Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 947403 Details for
Bug 1153444
vdsmd service down automatically and failed to restart
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
vdsm.log
vdsm.log (text/plain), 3.11 MB, created by
Liushihui
on 2014-10-16 04:05:43 UTC
(
hide
)
Description:
vdsm.log
Filename:
MIME Type:
Creator:
Liushihui
Created:
2014-10-16 04:05:43 UTC
Size:
3.11 MB
patch
obsolete
>JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:27,021::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:27,022::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3277::DEBUG::2014-10-15 06:37:27,022::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:27,024::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:27,024::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3278::DEBUG::2014-10-15 06:37:27,025::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3278::DEBUG::2014-10-15 06:37:27,025::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:37:27,769::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:27,791::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166644 s, 61.4 MB/s\n'; <rc> = 0 >Dummy-838::DEBUG::2014-10-15 06:37:29,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:29,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.014805 s, 69.2 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:30,036::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:30,036::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:30,037::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:30,114::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:30,114::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3279::DEBUG::2014-10-15 06:37:30,115::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:30,117::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:30,117::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3280::DEBUG::2014-10-15 06:37:30,118::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3280::DEBUG::2014-10-15 06:37:30,118::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:37:31,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:31,849::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0146239 s, 70.0 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:37:32,986::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:37:32,986::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:37:32,987::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:37:32,987::__init__::232::IOProcess::(_processLogs) (2427) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:37:32,987::__init__::232::IOProcess::(_processLogs) (2427) Queuing response >Thread-24::DEBUG::2014-10-15 06:37:32,987::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:37:32,993::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000403546 s, 1.1 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:37:32,993::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:37:32,994::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:37:32,994::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:37:32,994::__init__::232::IOProcess::(_processLogs) (2428) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:37:32,994::__init__::232::IOProcess::(_processLogs) (2428) Queuing response >Thread-21::DEBUG::2014-10-15 06:37:32,994::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:37:32,994::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:37:32,995::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:37:32,995::__init__::232::IOProcess::(_processLogs) (2429) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:37:32,995::__init__::232::IOProcess::(_processLogs) (2429) Queuing response >Thread-21::DEBUG::2014-10-15 06:37:32,995::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:37:32,995::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:37:32,995::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:37:32,995::__init__::232::IOProcess::(_processLogs) (2430) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:37:32,995::__init__::232::IOProcess::(_processLogs) (2430) Queuing response >Thread-21::DEBUG::2014-10-15 06:37:32,996::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:37:32,996::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:37:32,996::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:37:32,996::__init__::232::IOProcess::(_processLogs) (2431) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:37:32,996::__init__::232::IOProcess::(_processLogs) (2431) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:33,146::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:33,146::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3281::DEBUG::2014-10-15 06:37:33,147::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:33,148::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:33,149::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3282::DEBUG::2014-10-15 06:37:33,150::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3282::DEBUG::2014-10-15 06:37:33,150::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:37:33,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:33,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114891 s, 89.1 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:35,042::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:35,043::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:35,043::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:37:35,883::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:35,900::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115399 s, 88.7 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:36,174::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:36,175::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3283::DEBUG::2014-10-15 06:37:36,176::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:36,177::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:36,178::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3284::DEBUG::2014-10-15 06:37:36,178::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3284::DEBUG::2014-10-15 06:37:36,179::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:36,685::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-3285::DEBUG::2014-10-15 06:37:36,685::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >JsonRpcServer::DEBUG::2014-10-15 06:37:36,685::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3285::DEBUG::2014-10-15 06:37:36,687::task::595::Storage.TaskManager.Task::(_updateState) Task=`f65a62eb-f216-40e4-bd63-928f6c6d84dc`::moving from state init -> state preparing >Thread-3285::INFO::2014-10-15 06:37:36,687::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3285::INFO::2014-10-15 06:37:36,699::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3285::DEBUG::2014-10-15 06:37:36,699::task::1191::Storage.TaskManager.Task::(prepare) Task=`f65a62eb-f216-40e4-bd63-928f6c6d84dc`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3285::DEBUG::2014-10-15 06:37:36,699::task::595::Storage.TaskManager.Task::(_updateState) Task=`f65a62eb-f216-40e4-bd63-928f6c6d84dc`::moving from state preparing -> state finished >Thread-3285::DEBUG::2014-10-15 06:37:36,699::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3285::DEBUG::2014-10-15 06:37:36,699::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3285::DEBUG::2014-10-15 06:37:36,699::task::993::Storage.TaskManager.Task::(_decref) Task=`f65a62eb-f216-40e4-bd63-928f6c6d84dc`::ref 0 aborting False >Thread-3285::DEBUG::2014-10-15 06:37:36,699::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3285::DEBUG::2014-10-15 06:37:36,699::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:36,707::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:36,708::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3286::DEBUG::2014-10-15 06:37:36,708::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3286::DEBUG::2014-10-15 06:37:36,709::task::595::Storage.TaskManager.Task::(_updateState) Task=`2bbff11a-35d5-4afc-a018-d69d85139e69`::moving from state init -> state preparing >Thread-3286::INFO::2014-10-15 06:37:36,709::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3286::DEBUG::2014-10-15 06:37:36,709::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`d4561449-2dd4-48f9-974e-000a9d4d2be2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3286::DEBUG::2014-10-15 06:37:36,709::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3286::DEBUG::2014-10-15 06:37:36,709::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3286::DEBUG::2014-10-15 06:37:36,709::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`d4561449-2dd4-48f9-974e-000a9d4d2be2`::Granted request >Thread-3286::DEBUG::2014-10-15 06:37:36,710::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2bbff11a-35d5-4afc-a018-d69d85139e69`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3286::DEBUG::2014-10-15 06:37:36,710::task::993::Storage.TaskManager.Task::(_decref) Task=`2bbff11a-35d5-4afc-a018-d69d85139e69`::ref 1 aborting False >Thread-3286::INFO::2014-10-15 06:37:36,728::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3286::DEBUG::2014-10-15 06:37:36,728::task::1191::Storage.TaskManager.Task::(prepare) Task=`2bbff11a-35d5-4afc-a018-d69d85139e69`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3286::DEBUG::2014-10-15 06:37:36,728::task::595::Storage.TaskManager.Task::(_updateState) Task=`2bbff11a-35d5-4afc-a018-d69d85139e69`::moving from state preparing -> state finished >Thread-3286::DEBUG::2014-10-15 06:37:36,728::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3286::DEBUG::2014-10-15 06:37:36,728::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3286::DEBUG::2014-10-15 06:37:36,728::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3286::DEBUG::2014-10-15 06:37:36,728::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3286::DEBUG::2014-10-15 06:37:36,728::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3286::DEBUG::2014-10-15 06:37:36,729::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3286::DEBUG::2014-10-15 06:37:36,729::task::993::Storage.TaskManager.Task::(_decref) Task=`2bbff11a-35d5-4afc-a018-d69d85139e69`::ref 0 aborting False >Thread-3286::DEBUG::2014-10-15 06:37:36,729::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3286::DEBUG::2014-10-15 06:37:36,729::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:37:37,254::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 47. >Dummy-838::DEBUG::2014-10-15 06:37:37,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:37,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114909 s, 89.1 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:39,199::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:39,200::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3287::DEBUG::2014-10-15 06:37:39,201::task::595::Storage.TaskManager.Task::(_updateState) Task=`885d0ffe-e9b6-4f10-9be5-4554b6424ac3`::moving from state init -> state preparing >Thread-3287::INFO::2014-10-15 06:37:39,201::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-3287::INFO::2014-10-15 06:37:39,201::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000403546', 'lastCheck': '6.2', 'valid': True}} >Thread-3287::DEBUG::2014-10-15 06:37:39,202::task::1191::Storage.TaskManager.Task::(prepare) Task=`885d0ffe-e9b6-4f10-9be5-4554b6424ac3`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000403546', 'lastCheck': '6.2', 'valid': True}} >Thread-3287::DEBUG::2014-10-15 06:37:39,202::task::595::Storage.TaskManager.Task::(_updateState) Task=`885d0ffe-e9b6-4f10-9be5-4554b6424ac3`::moving from state preparing -> state finished >Thread-3287::DEBUG::2014-10-15 06:37:39,202::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3287::DEBUG::2014-10-15 06:37:39,202::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3287::DEBUG::2014-10-15 06:37:39,202::task::993::Storage.TaskManager.Task::(_decref) Task=`885d0ffe-e9b6-4f10-9be5-4554b6424ac3`::ref 0 aborting False >Thread-3287::ERROR::2014-10-15 06:37:39,203::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-3287::DEBUG::2014-10-15 06:37:39,203::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:39,212::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:39,212::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3288::DEBUG::2014-10-15 06:37:39,213::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3288::DEBUG::2014-10-15 06:37:39,214::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:37:39,935::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:39,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114975 s, 89.1 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:40,046::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:40,047::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:40,047::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:37:41,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:41,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114881 s, 89.1 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:42,324::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:42,325::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3289::DEBUG::2014-10-15 06:37:42,326::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:42,327::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:42,328::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3290::DEBUG::2014-10-15 06:37:42,328::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3290::DEBUG::2014-10-15 06:37:42,329::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-21::DEBUG::2014-10-15 06:37:42,997::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:37:42,997::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:37:42,997::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:37:42,998::__init__::232::IOProcess::(_processLogs) (2432) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:37:42,998::__init__::232::IOProcess::(_processLogs) (2432) Queuing response >Thread-24::DEBUG::2014-10-15 06:37:42,998::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:37:43,004::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000388242 s, 1.2 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:37:43,004::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:37:43,005::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:37:43,005::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:37:43,005::__init__::232::IOProcess::(_processLogs) (2433) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:37:43,005::__init__::232::IOProcess::(_processLogs) (2433) Queuing response >Thread-21::DEBUG::2014-10-15 06:37:43,005::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:37:43,006::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:37:43,006::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:37:43,006::__init__::232::IOProcess::(_processLogs) (2434) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:37:43,006::__init__::232::IOProcess::(_processLogs) (2434) Queuing response >Thread-21::DEBUG::2014-10-15 06:37:43,006::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:37:43,006::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:37:43,006::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:37:43,006::__init__::232::IOProcess::(_processLogs) (2435) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:37:43,006::__init__::232::IOProcess::(_processLogs) (2435) Queuing response >Thread-21::DEBUG::2014-10-15 06:37:43,007::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:37:43,007::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:37:43,007::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:37:43,007::__init__::232::IOProcess::(_processLogs) (2436) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:37:43,007::__init__::232::IOProcess::(_processLogs) (2436) Queuing response >Dummy-838::DEBUG::2014-10-15 06:37:43,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:44,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167933 s, 61.0 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:45,052::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:45,052::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:45,053::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:45,365::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:45,366::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3291::DEBUG::2014-10-15 06:37:45,367::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:45,368::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:45,368::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3292::DEBUG::2014-10-15 06:37:45,369::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3292::DEBUG::2014-10-15 06:37:45,369::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:37:46,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:46,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117214 s, 87.4 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:46,746::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:46,747::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3293::DEBUG::2014-10-15 06:37:46,747::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3293::DEBUG::2014-10-15 06:37:46,748::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e43142b-4969-464d-8e60-a995678535fd`::moving from state init -> state preparing >Thread-3293::INFO::2014-10-15 06:37:46,748::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3293::INFO::2014-10-15 06:37:46,760::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3293::DEBUG::2014-10-15 06:37:46,760::task::1191::Storage.TaskManager.Task::(prepare) Task=`7e43142b-4969-464d-8e60-a995678535fd`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3293::DEBUG::2014-10-15 06:37:46,760::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e43142b-4969-464d-8e60-a995678535fd`::moving from state preparing -> state finished >Thread-3293::DEBUG::2014-10-15 06:37:46,760::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3293::DEBUG::2014-10-15 06:37:46,760::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3293::DEBUG::2014-10-15 06:37:46,760::task::993::Storage.TaskManager.Task::(_decref) Task=`7e43142b-4969-464d-8e60-a995678535fd`::ref 0 aborting False >Thread-3293::DEBUG::2014-10-15 06:37:46,760::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3293::DEBUG::2014-10-15 06:37:46,760::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:46,768::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:46,769::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3294::DEBUG::2014-10-15 06:37:46,769::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3294::DEBUG::2014-10-15 06:37:46,769::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb1ba55d-0c1f-464c-b8e5-5bdceda371a3`::moving from state init -> state preparing >Thread-3294::INFO::2014-10-15 06:37:46,769::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3294::DEBUG::2014-10-15 06:37:46,770::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`4464f5d5-2c85-498f-9fde-55e8af39c3b3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3294::DEBUG::2014-10-15 06:37:46,771::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3294::DEBUG::2014-10-15 06:37:46,771::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3294::DEBUG::2014-10-15 06:37:46,771::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`4464f5d5-2c85-498f-9fde-55e8af39c3b3`::Granted request >Thread-3294::DEBUG::2014-10-15 06:37:46,771::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fb1ba55d-0c1f-464c-b8e5-5bdceda371a3`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3294::DEBUG::2014-10-15 06:37:46,771::task::993::Storage.TaskManager.Task::(_decref) Task=`fb1ba55d-0c1f-464c-b8e5-5bdceda371a3`::ref 1 aborting False >Thread-3294::INFO::2014-10-15 06:37:46,789::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3294::DEBUG::2014-10-15 06:37:46,789::task::1191::Storage.TaskManager.Task::(prepare) Task=`fb1ba55d-0c1f-464c-b8e5-5bdceda371a3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3294::DEBUG::2014-10-15 06:37:46,789::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb1ba55d-0c1f-464c-b8e5-5bdceda371a3`::moving from state preparing -> state finished >Thread-3294::DEBUG::2014-10-15 06:37:46,790::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3294::DEBUG::2014-10-15 06:37:46,790::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3294::DEBUG::2014-10-15 06:37:46,790::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3294::DEBUG::2014-10-15 06:37:46,790::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3294::DEBUG::2014-10-15 06:37:46,790::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3294::DEBUG::2014-10-15 06:37:46,790::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3294::DEBUG::2014-10-15 06:37:46,790::task::993::Storage.TaskManager.Task::(_decref) Task=`fb1ba55d-0c1f-464c-b8e5-5bdceda371a3`::ref 0 aborting False >Thread-3294::DEBUG::2014-10-15 06:37:46,790::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3294::DEBUG::2014-10-15 06:37:46,791::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:37:48,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:48,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114891 s, 89.1 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:48,390::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:48,391::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3295::DEBUG::2014-10-15 06:37:48,392::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:48,393::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:48,393::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3296::DEBUG::2014-10-15 06:37:48,394::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3296::DEBUG::2014-10-15 06:37:48,395::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:50,058::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:50,059::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:50,059::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:37:50,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:50,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116658 s, 87.8 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:51,411::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:51,411::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3297::DEBUG::2014-10-15 06:37:51,412::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:51,414::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:51,414::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3298::DEBUG::2014-10-15 06:37:51,415::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3298::DEBUG::2014-10-15 06:37:51,415::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:37:52,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:52,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0143548 s, 71.3 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:37:53,008::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:37:53,008::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:37:53,009::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:37:53,009::__init__::232::IOProcess::(_processLogs) (2437) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:37:53,009::__init__::232::IOProcess::(_processLogs) (2437) Queuing response >Thread-24::DEBUG::2014-10-15 06:37:53,009::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:37:53,015::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000376372 s, 1.2 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:37:53,016::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:37:53,016::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:37:53,016::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:37:53,016::__init__::232::IOProcess::(_processLogs) (2438) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:37:53,016::__init__::232::IOProcess::(_processLogs) (2438) Queuing response >Thread-21::DEBUG::2014-10-15 06:37:53,017::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:37:53,017::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:37:53,017::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:37:53,017::__init__::232::IOProcess::(_processLogs) (2439) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:37:53,017::__init__::232::IOProcess::(_processLogs) (2439) Queuing response >Thread-21::DEBUG::2014-10-15 06:37:53,017::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:37:53,018::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:37:53,018::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:37:53,018::__init__::232::IOProcess::(_processLogs) (2440) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:37:53,018::__init__::232::IOProcess::(_processLogs) (2440) Queuing response >Thread-21::DEBUG::2014-10-15 06:37:53,018::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:37:53,018::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:37:53,018::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:37:53,018::__init__::232::IOProcess::(_processLogs) (2441) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:37:53,018::__init__::232::IOProcess::(_processLogs) (2441) Queuing response >Dummy-838::DEBUG::2014-10-15 06:37:54,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:54,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131847 s, 77.7 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:54,437::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:54,437::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3299::DEBUG::2014-10-15 06:37:54,439::task::595::Storage.TaskManager.Task::(_updateState) Task=`b876f6b9-8ed2-4ec6-abc3-99aea6568791`::moving from state init -> state preparing >Thread-3299::INFO::2014-10-15 06:37:54,439::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-3299::INFO::2014-10-15 06:37:54,439::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000376372', 'lastCheck': '1.4', 'valid': True}} >Thread-3299::DEBUG::2014-10-15 06:37:54,439::task::1191::Storage.TaskManager.Task::(prepare) Task=`b876f6b9-8ed2-4ec6-abc3-99aea6568791`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000376372', 'lastCheck': '1.4', 'valid': True}} >Thread-3299::DEBUG::2014-10-15 06:37:54,439::task::595::Storage.TaskManager.Task::(_updateState) Task=`b876f6b9-8ed2-4ec6-abc3-99aea6568791`::moving from state preparing -> state finished >Thread-3299::DEBUG::2014-10-15 06:37:54,439::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3299::DEBUG::2014-10-15 06:37:54,439::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3299::DEBUG::2014-10-15 06:37:54,439::task::993::Storage.TaskManager.Task::(_decref) Task=`b876f6b9-8ed2-4ec6-abc3-99aea6568791`::ref 0 aborting False >Thread-3299::ERROR::2014-10-15 06:37:54,440::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-3299::DEBUG::2014-10-15 06:37:54,441::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:54,451::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:54,451::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3300::DEBUG::2014-10-15 06:37:54,452::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3300::DEBUG::2014-10-15 06:37:54,453::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:55,065::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:55,065::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:37:55,066::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:37:56,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:56,231::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116957 s, 87.6 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:56,809::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:56,810::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3301::DEBUG::2014-10-15 06:37:56,810::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3301::DEBUG::2014-10-15 06:37:56,811::task::595::Storage.TaskManager.Task::(_updateState) Task=`c4e98954-a876-4e6e-a4a4-c9e512d8a419`::moving from state init -> state preparing >Thread-3301::INFO::2014-10-15 06:37:56,811::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3301::INFO::2014-10-15 06:37:56,823::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3301::DEBUG::2014-10-15 06:37:56,823::task::1191::Storage.TaskManager.Task::(prepare) Task=`c4e98954-a876-4e6e-a4a4-c9e512d8a419`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3301::DEBUG::2014-10-15 06:37:56,823::task::595::Storage.TaskManager.Task::(_updateState) Task=`c4e98954-a876-4e6e-a4a4-c9e512d8a419`::moving from state preparing -> state finished >Thread-3301::DEBUG::2014-10-15 06:37:56,823::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3301::DEBUG::2014-10-15 06:37:56,823::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3301::DEBUG::2014-10-15 06:37:56,823::task::993::Storage.TaskManager.Task::(_decref) Task=`c4e98954-a876-4e6e-a4a4-c9e512d8a419`::ref 0 aborting False >Thread-3301::DEBUG::2014-10-15 06:37:56,823::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3301::DEBUG::2014-10-15 06:37:56,823::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:56,830::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:56,830::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3302::DEBUG::2014-10-15 06:37:56,830::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3302::DEBUG::2014-10-15 06:37:56,831::task::595::Storage.TaskManager.Task::(_updateState) Task=`711d374c-4c67-41ce-a42c-0e573f4f347d`::moving from state init -> state preparing >Thread-3302::INFO::2014-10-15 06:37:56,831::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3302::DEBUG::2014-10-15 06:37:56,831::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`64e21a6d-fee8-4725-a683-9f6a8fa09148`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3302::DEBUG::2014-10-15 06:37:56,831::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3302::DEBUG::2014-10-15 06:37:56,832::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3302::DEBUG::2014-10-15 06:37:56,832::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`64e21a6d-fee8-4725-a683-9f6a8fa09148`::Granted request >Thread-3302::DEBUG::2014-10-15 06:37:56,832::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`711d374c-4c67-41ce-a42c-0e573f4f347d`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3302::DEBUG::2014-10-15 06:37:56,832::task::993::Storage.TaskManager.Task::(_decref) Task=`711d374c-4c67-41ce-a42c-0e573f4f347d`::ref 1 aborting False >Thread-3302::INFO::2014-10-15 06:37:56,850::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3302::DEBUG::2014-10-15 06:37:56,850::task::1191::Storage.TaskManager.Task::(prepare) Task=`711d374c-4c67-41ce-a42c-0e573f4f347d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3302::DEBUG::2014-10-15 06:37:56,850::task::595::Storage.TaskManager.Task::(_updateState) Task=`711d374c-4c67-41ce-a42c-0e573f4f347d`::moving from state preparing -> state finished >Thread-3302::DEBUG::2014-10-15 06:37:56,850::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3302::DEBUG::2014-10-15 06:37:56,851::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3302::DEBUG::2014-10-15 06:37:56,851::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3302::DEBUG::2014-10-15 06:37:56,851::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3302::DEBUG::2014-10-15 06:37:56,851::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3302::DEBUG::2014-10-15 06:37:56,851::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3302::DEBUG::2014-10-15 06:37:56,851::task::993::Storage.TaskManager.Task::(_decref) Task=`711d374c-4c67-41ce-a42c-0e573f4f347d`::ref 0 aborting False >Thread-3302::DEBUG::2014-10-15 06:37:56,851::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3302::DEBUG::2014-10-15 06:37:56,851::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:57,580::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:57,580::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3303::DEBUG::2014-10-15 06:37:57,581::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:37:57,582::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:37:57,583::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3304::DEBUG::2014-10-15 06:37:57,584::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3304::DEBUG::2014-10-15 06:37:57,584::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:37:58,239::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:37:58,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114248 s, 89.6 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:00,071::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:00,072::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:00,072::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:38:00,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:00,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114622 s, 89.3 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:00,613::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:00,614::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3305::DEBUG::2014-10-15 06:38:00,615::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:00,616::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:00,616::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3306::DEBUG::2014-10-15 06:38:00,617::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3306::DEBUG::2014-10-15 06:38:00,618::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:38:02,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:02,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011805 s, 86.7 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:38:03,020::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:03,020::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:03,020::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:03,020::__init__::232::IOProcess::(_processLogs) (2442) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:38:03,020::__init__::232::IOProcess::(_processLogs) (2442) Queuing response >Thread-24::DEBUG::2014-10-15 06:38:03,020::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:38:03,026::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000391792 s, 1.2 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:38:03,027::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:03,027::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:03,027::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:03,027::__init__::232::IOProcess::(_processLogs) (2443) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:38:03,027::__init__::232::IOProcess::(_processLogs) (2443) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:03,028::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:03,028::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:03,028::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:03,028::__init__::232::IOProcess::(_processLogs) (2444) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:03,028::__init__::232::IOProcess::(_processLogs) (2444) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:03,028::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:03,029::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:03,029::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:03,029::__init__::232::IOProcess::(_processLogs) (2445) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:03,029::__init__::232::IOProcess::(_processLogs) (2445) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:03,029::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:03,029::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:03,029::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:03,030::__init__::232::IOProcess::(_processLogs) (2446) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:03,030::__init__::232::IOProcess::(_processLogs) (2446) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:03,650::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:03,650::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3307::DEBUG::2014-10-15 06:38:03,651::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:03,652::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:03,653::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3308::DEBUG::2014-10-15 06:38:03,654::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3308::DEBUG::2014-10-15 06:38:03,654::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:38:04,318::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:04,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117643 s, 87.0 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:05,079::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-167::DEBUG::2014-10-15 06:38:05,079::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8231498-0812-47c2-9fe4-e24fa1445b1d`::moving from state init -> state preparing >Thread-167::INFO::2014-10-15 06:38:05,080::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:05,081::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-21::DEBUG::2014-10-15 06:38:05,082::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:05,082::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:05,083::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:05,083::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-21::DEBUG::2014-10-15 06:38:05,083::__init__::232::IOProcess::(_processLogs) (2447) Got request for method 'stat' >Thread-21::DEBUG::2014-10-15 06:38:05,084::__init__::232::IOProcess::(_processLogs) (2447) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:05,085::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:05,085::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:05,085::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:05,085::__init__::232::IOProcess::(_processLogs) (2448) Got request for method 'stat' >Thread-21::DEBUG::2014-10-15 06:38:05,086::__init__::232::IOProcess::(_processLogs) (2448) Queuing response >Thread-167::INFO::2014-10-15 06:38:05,086::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-167::DEBUG::2014-10-15 06:38:05,086::task::1191::Storage.TaskManager.Task::(prepare) Task=`e8231498-0812-47c2-9fe4-e24fa1445b1d`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-167::DEBUG::2014-10-15 06:38:05,086::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8231498-0812-47c2-9fe4-e24fa1445b1d`::moving from state preparing -> state finished >Thread-167::DEBUG::2014-10-15 06:38:05,086::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-167::DEBUG::2014-10-15 06:38:05,086::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-167::DEBUG::2014-10-15 06:38:05,086::task::993::Storage.TaskManager.Task::(_decref) Task=`e8231498-0812-47c2-9fe4-e24fa1445b1d`::ref 0 aborting False >Dummy-838::DEBUG::2014-10-15 06:38:06,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:06,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114398 s, 89.5 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:06,675::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:06,676::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3309::DEBUG::2014-10-15 06:38:06,677::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:06,678::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:06,679::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3310::DEBUG::2014-10-15 06:38:06,680::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3310::DEBUG::2014-10-15 06:38:06,680::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:06,868::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:06,869::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3311::DEBUG::2014-10-15 06:38:06,869::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3311::DEBUG::2014-10-15 06:38:06,870::task::595::Storage.TaskManager.Task::(_updateState) Task=`d795654a-cd34-4dc7-8642-ad83e5c40c06`::moving from state init -> state preparing >Thread-3311::INFO::2014-10-15 06:38:06,870::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3311::INFO::2014-10-15 06:38:06,881::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3311::DEBUG::2014-10-15 06:38:06,882::task::1191::Storage.TaskManager.Task::(prepare) Task=`d795654a-cd34-4dc7-8642-ad83e5c40c06`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3311::DEBUG::2014-10-15 06:38:06,882::task::595::Storage.TaskManager.Task::(_updateState) Task=`d795654a-cd34-4dc7-8642-ad83e5c40c06`::moving from state preparing -> state finished >Thread-3311::DEBUG::2014-10-15 06:38:06,882::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3311::DEBUG::2014-10-15 06:38:06,882::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3311::DEBUG::2014-10-15 06:38:06,882::task::993::Storage.TaskManager.Task::(_decref) Task=`d795654a-cd34-4dc7-8642-ad83e5c40c06`::ref 0 aborting False >Thread-3311::DEBUG::2014-10-15 06:38:06,882::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3311::DEBUG::2014-10-15 06:38:06,882::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:06,890::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:06,891::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3312::DEBUG::2014-10-15 06:38:06,891::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3312::DEBUG::2014-10-15 06:38:06,891::task::595::Storage.TaskManager.Task::(_updateState) Task=`a2911547-8a33-44a4-9d72-5489b6ba5d0e`::moving from state init -> state preparing >Thread-3312::INFO::2014-10-15 06:38:06,891::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3312::DEBUG::2014-10-15 06:38:06,892::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`dcdef5bd-e6f8-4d9b-8085-7bca87626bb6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3312::DEBUG::2014-10-15 06:38:06,892::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3312::DEBUG::2014-10-15 06:38:06,892::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3312::DEBUG::2014-10-15 06:38:06,892::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`dcdef5bd-e6f8-4d9b-8085-7bca87626bb6`::Granted request >Thread-3312::DEBUG::2014-10-15 06:38:06,892::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a2911547-8a33-44a4-9d72-5489b6ba5d0e`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3312::DEBUG::2014-10-15 06:38:06,892::task::993::Storage.TaskManager.Task::(_decref) Task=`a2911547-8a33-44a4-9d72-5489b6ba5d0e`::ref 1 aborting False >Thread-3312::INFO::2014-10-15 06:38:06,911::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3312::DEBUG::2014-10-15 06:38:06,911::task::1191::Storage.TaskManager.Task::(prepare) Task=`a2911547-8a33-44a4-9d72-5489b6ba5d0e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3312::DEBUG::2014-10-15 06:38:06,911::task::595::Storage.TaskManager.Task::(_updateState) Task=`a2911547-8a33-44a4-9d72-5489b6ba5d0e`::moving from state preparing -> state finished >Thread-3312::DEBUG::2014-10-15 06:38:06,911::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3312::DEBUG::2014-10-15 06:38:06,911::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3312::DEBUG::2014-10-15 06:38:06,911::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3312::DEBUG::2014-10-15 06:38:06,911::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3312::DEBUG::2014-10-15 06:38:06,911::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3312::DEBUG::2014-10-15 06:38:06,911::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3312::DEBUG::2014-10-15 06:38:06,912::task::993::Storage.TaskManager.Task::(_decref) Task=`a2911547-8a33-44a4-9d72-5489b6ba5d0e`::ref 0 aborting False >Thread-3312::DEBUG::2014-10-15 06:38:06,912::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3312::DEBUG::2014-10-15 06:38:06,912::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:38:07,287::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 47. >Dummy-838::DEBUG::2014-10-15 06:38:08,369::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:08,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114685 s, 89.3 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:09,709::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:09,709::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3313::DEBUG::2014-10-15 06:38:09,710::task::595::Storage.TaskManager.Task::(_updateState) Task=`c5bced11-0284-41a5-9637-529888e2d00e`::moving from state init -> state preparing >Thread-3313::INFO::2014-10-15 06:38:09,711::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-3313::INFO::2014-10-15 06:38:09,711::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000391792', 'lastCheck': '6.7', 'valid': True}} >Thread-3313::DEBUG::2014-10-15 06:38:09,711::task::1191::Storage.TaskManager.Task::(prepare) Task=`c5bced11-0284-41a5-9637-529888e2d00e`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000391792', 'lastCheck': '6.7', 'valid': True}} >Thread-3313::DEBUG::2014-10-15 06:38:09,711::task::595::Storage.TaskManager.Task::(_updateState) Task=`c5bced11-0284-41a5-9637-529888e2d00e`::moving from state preparing -> state finished >Thread-3313::DEBUG::2014-10-15 06:38:09,711::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3313::DEBUG::2014-10-15 06:38:09,711::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3313::DEBUG::2014-10-15 06:38:09,711::task::993::Storage.TaskManager.Task::(_decref) Task=`c5bced11-0284-41a5-9637-529888e2d00e`::ref 0 aborting False >Thread-3313::ERROR::2014-10-15 06:38:09,712::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-3313::DEBUG::2014-10-15 06:38:09,713::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:09,720::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:09,720::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3314::DEBUG::2014-10-15 06:38:09,721::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3314::DEBUG::2014-10-15 06:38:09,722::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:10,088::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:10,088::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:10,089::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:38:10,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:10,412::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114415 s, 89.5 MB/s\n'; <rc> = 0 >Dummy-838::DEBUG::2014-10-15 06:38:12,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:12,438::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011414 s, 89.7 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:12,846::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:12,847::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3315::DEBUG::2014-10-15 06:38:12,848::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:12,850::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:12,850::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3316::DEBUG::2014-10-15 06:38:12,851::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3316::DEBUG::2014-10-15 06:38:12,851::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-21::DEBUG::2014-10-15 06:38:13,031::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:13,031::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:13,031::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:13,031::__init__::232::IOProcess::(_processLogs) (2449) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:38:13,031::__init__::232::IOProcess::(_processLogs) (2449) Queuing response >Thread-24::DEBUG::2014-10-15 06:38:13,031::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:38:13,037::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00034775 s, 1.3 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:38:13,038::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:13,038::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:13,038::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:13,038::__init__::232::IOProcess::(_processLogs) (2450) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:38:13,038::__init__::232::IOProcess::(_processLogs) (2450) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:13,039::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:13,039::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:13,039::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:13,039::__init__::232::IOProcess::(_processLogs) (2451) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:13,039::__init__::232::IOProcess::(_processLogs) (2451) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:13,039::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:13,039::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:13,040::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:13,040::__init__::232::IOProcess::(_processLogs) (2452) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:13,040::__init__::232::IOProcess::(_processLogs) (2452) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:13,040::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:13,040::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:13,040::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:13,040::__init__::232::IOProcess::(_processLogs) (2453) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:13,040::__init__::232::IOProcess::(_processLogs) (2453) Queuing response >Dummy-838::DEBUG::2014-10-15 06:38:14,447::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:14,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163866 s, 62.5 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:15,095::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:15,095::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:15,096::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:15,877::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:15,878::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3317::DEBUG::2014-10-15 06:38:15,879::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:15,880::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:15,881::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3318::DEBUG::2014-10-15 06:38:15,882::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3318::DEBUG::2014-10-15 06:38:15,882::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:38:16,543::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:16,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178079 s, 57.5 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:16,948::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-3319::DEBUG::2014-10-15 06:38:16,948::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >JsonRpcServer::DEBUG::2014-10-15 06:38:16,948::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3319::DEBUG::2014-10-15 06:38:16,949::task::595::Storage.TaskManager.Task::(_updateState) Task=`d419a47c-0a40-452d-a18d-fd8f5694777d`::moving from state init -> state preparing >Thread-3319::INFO::2014-10-15 06:38:16,950::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3319::INFO::2014-10-15 06:38:16,961::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3319::DEBUG::2014-10-15 06:38:16,961::task::1191::Storage.TaskManager.Task::(prepare) Task=`d419a47c-0a40-452d-a18d-fd8f5694777d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3319::DEBUG::2014-10-15 06:38:16,961::task::595::Storage.TaskManager.Task::(_updateState) Task=`d419a47c-0a40-452d-a18d-fd8f5694777d`::moving from state preparing -> state finished >Thread-3319::DEBUG::2014-10-15 06:38:16,961::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3319::DEBUG::2014-10-15 06:38:16,962::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3319::DEBUG::2014-10-15 06:38:16,962::task::993::Storage.TaskManager.Task::(_decref) Task=`d419a47c-0a40-452d-a18d-fd8f5694777d`::ref 0 aborting False >Thread-3319::DEBUG::2014-10-15 06:38:16,962::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3319::DEBUG::2014-10-15 06:38:16,962::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:16,969::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:16,969::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3320::DEBUG::2014-10-15 06:38:16,969::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3320::DEBUG::2014-10-15 06:38:16,970::task::595::Storage.TaskManager.Task::(_updateState) Task=`06fde4a4-cbd3-45af-83aa-6f08aa003f2c`::moving from state init -> state preparing >Thread-3320::INFO::2014-10-15 06:38:16,970::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3320::DEBUG::2014-10-15 06:38:16,970::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`a7761122-789b-4eba-a3bc-8a513aa13533`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3320::DEBUG::2014-10-15 06:38:16,970::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3320::DEBUG::2014-10-15 06:38:16,971::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3320::DEBUG::2014-10-15 06:38:16,971::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`a7761122-789b-4eba-a3bc-8a513aa13533`::Granted request >Thread-3320::DEBUG::2014-10-15 06:38:16,971::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`06fde4a4-cbd3-45af-83aa-6f08aa003f2c`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3320::DEBUG::2014-10-15 06:38:16,971::task::993::Storage.TaskManager.Task::(_decref) Task=`06fde4a4-cbd3-45af-83aa-6f08aa003f2c`::ref 1 aborting False >Thread-3320::INFO::2014-10-15 06:38:16,989::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3320::DEBUG::2014-10-15 06:38:16,989::task::1191::Storage.TaskManager.Task::(prepare) Task=`06fde4a4-cbd3-45af-83aa-6f08aa003f2c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3320::DEBUG::2014-10-15 06:38:16,989::task::595::Storage.TaskManager.Task::(_updateState) Task=`06fde4a4-cbd3-45af-83aa-6f08aa003f2c`::moving from state preparing -> state finished >Thread-3320::DEBUG::2014-10-15 06:38:16,989::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3320::DEBUG::2014-10-15 06:38:16,989::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3320::DEBUG::2014-10-15 06:38:16,989::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3320::DEBUG::2014-10-15 06:38:16,989::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3320::DEBUG::2014-10-15 06:38:16,989::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3320::DEBUG::2014-10-15 06:38:16,989::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3320::DEBUG::2014-10-15 06:38:16,990::task::993::Storage.TaskManager.Task::(_decref) Task=`06fde4a4-cbd3-45af-83aa-6f08aa003f2c`::ref 0 aborting False >Thread-3320::DEBUG::2014-10-15 06:38:16,990::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3320::DEBUG::2014-10-15 06:38:16,990::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:38:18,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:18,597::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172254 s, 59.4 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:18,905::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:18,906::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3321::DEBUG::2014-10-15 06:38:18,907::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:18,908::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:18,909::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3322::DEBUG::2014-10-15 06:38:18,910::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3322::DEBUG::2014-10-15 06:38:18,910::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:20,101::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:20,102::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:20,102::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:38:20,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:20,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117507 s, 87.1 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:21,935::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:21,936::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3323::DEBUG::2014-10-15 06:38:21,937::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:21,939::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:21,939::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3324::DEBUG::2014-10-15 06:38:21,940::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3324::DEBUG::2014-10-15 06:38:21,940::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:38:22,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:22,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011339 s, 90.3 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:38:23,041::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:23,042::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:23,042::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:23,042::__init__::232::IOProcess::(_processLogs) (2454) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:38:23,042::__init__::232::IOProcess::(_processLogs) (2454) Queuing response >Thread-24::DEBUG::2014-10-15 06:38:23,043::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:38:23,049::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000435472 s, 1.1 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:38:23,049::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:23,049::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:23,050::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:23,050::__init__::232::IOProcess::(_processLogs) (2455) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:38:23,050::__init__::232::IOProcess::(_processLogs) (2455) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:23,050::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:23,050::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:23,051::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:23,051::__init__::232::IOProcess::(_processLogs) (2456) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:23,051::__init__::232::IOProcess::(_processLogs) (2456) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:23,051::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:23,051::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:23,051::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:23,051::__init__::232::IOProcess::(_processLogs) (2457) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:23,051::__init__::232::IOProcess::(_processLogs) (2457) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:23,052::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:23,052::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:23,052::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:23,052::__init__::232::IOProcess::(_processLogs) (2458) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:23,052::__init__::232::IOProcess::(_processLogs) (2458) Queuing response >Dummy-838::DEBUG::2014-10-15 06:38:24,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:24,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0152977 s, 66.9 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:24,963::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:24,964::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3325::DEBUG::2014-10-15 06:38:24,965::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce3ffdbc-fb5b-4cfb-ac45-9dad089e5576`::moving from state init -> state preparing >Thread-3325::INFO::2014-10-15 06:38:24,965::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-3325::INFO::2014-10-15 06:38:24,965::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000435472', 'lastCheck': '1.9', 'valid': True}} >Thread-3325::DEBUG::2014-10-15 06:38:24,966::task::1191::Storage.TaskManager.Task::(prepare) Task=`ce3ffdbc-fb5b-4cfb-ac45-9dad089e5576`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000435472', 'lastCheck': '1.9', 'valid': True}} >Thread-3325::DEBUG::2014-10-15 06:38:24,966::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce3ffdbc-fb5b-4cfb-ac45-9dad089e5576`::moving from state preparing -> state finished >Thread-3325::DEBUG::2014-10-15 06:38:24,966::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3325::DEBUG::2014-10-15 06:38:24,966::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3325::DEBUG::2014-10-15 06:38:24,966::task::993::Storage.TaskManager.Task::(_decref) Task=`ce3ffdbc-fb5b-4cfb-ac45-9dad089e5576`::ref 0 aborting False >Thread-3325::ERROR::2014-10-15 06:38:24,967::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-3325::DEBUG::2014-10-15 06:38:24,967::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:24,979::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:24,979::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3326::DEBUG::2014-10-15 06:38:24,980::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3326::DEBUG::2014-10-15 06:38:24,981::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:25,108::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:25,108::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:25,109::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:38:26,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:26,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113346 s, 90.3 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:27,008::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:27,009::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3327::DEBUG::2014-10-15 06:38:27,009::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3327::DEBUG::2014-10-15 06:38:27,010::task::595::Storage.TaskManager.Task::(_updateState) Task=`fa84b4f5-28fa-4d5b-bd5a-f631ec4ce721`::moving from state init -> state preparing >Thread-3327::INFO::2014-10-15 06:38:27,010::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3327::INFO::2014-10-15 06:38:27,022::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3327::DEBUG::2014-10-15 06:38:27,023::task::1191::Storage.TaskManager.Task::(prepare) Task=`fa84b4f5-28fa-4d5b-bd5a-f631ec4ce721`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3327::DEBUG::2014-10-15 06:38:27,023::task::595::Storage.TaskManager.Task::(_updateState) Task=`fa84b4f5-28fa-4d5b-bd5a-f631ec4ce721`::moving from state preparing -> state finished >Thread-3327::DEBUG::2014-10-15 06:38:27,023::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3327::DEBUG::2014-10-15 06:38:27,023::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3327::DEBUG::2014-10-15 06:38:27,023::task::993::Storage.TaskManager.Task::(_decref) Task=`fa84b4f5-28fa-4d5b-bd5a-f631ec4ce721`::ref 0 aborting False >Thread-3327::DEBUG::2014-10-15 06:38:27,023::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3327::DEBUG::2014-10-15 06:38:27,023::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:27,029::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:27,030::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3328::DEBUG::2014-10-15 06:38:27,030::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3328::DEBUG::2014-10-15 06:38:27,031::task::595::Storage.TaskManager.Task::(_updateState) Task=`7df524ba-1641-4f55-8dd3-4212ef0def2d`::moving from state init -> state preparing >Thread-3328::INFO::2014-10-15 06:38:27,031::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3328::DEBUG::2014-10-15 06:38:27,031::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`797cce92-e413-42bf-a0b9-d90eb3c4fb40`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3328::DEBUG::2014-10-15 06:38:27,031::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3328::DEBUG::2014-10-15 06:38:27,032::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3328::DEBUG::2014-10-15 06:38:27,032::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`797cce92-e413-42bf-a0b9-d90eb3c4fb40`::Granted request >Thread-3328::DEBUG::2014-10-15 06:38:27,033::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7df524ba-1641-4f55-8dd3-4212ef0def2d`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3328::DEBUG::2014-10-15 06:38:27,033::task::993::Storage.TaskManager.Task::(_decref) Task=`7df524ba-1641-4f55-8dd3-4212ef0def2d`::ref 1 aborting False >Thread-3328::INFO::2014-10-15 06:38:27,051::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3328::DEBUG::2014-10-15 06:38:27,051::task::1191::Storage.TaskManager.Task::(prepare) Task=`7df524ba-1641-4f55-8dd3-4212ef0def2d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3328::DEBUG::2014-10-15 06:38:27,051::task::595::Storage.TaskManager.Task::(_updateState) Task=`7df524ba-1641-4f55-8dd3-4212ef0def2d`::moving from state preparing -> state finished >Thread-3328::DEBUG::2014-10-15 06:38:27,051::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3328::DEBUG::2014-10-15 06:38:27,051::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3328::DEBUG::2014-10-15 06:38:27,052::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3328::DEBUG::2014-10-15 06:38:27,052::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3328::DEBUG::2014-10-15 06:38:27,052::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3328::DEBUG::2014-10-15 06:38:27,052::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3328::DEBUG::2014-10-15 06:38:27,052::task::993::Storage.TaskManager.Task::(_decref) Task=`7df524ba-1641-4f55-8dd3-4212ef0def2d`::ref 0 aborting False >Thread-3328::DEBUG::2014-10-15 06:38:27,052::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3328::DEBUG::2014-10-15 06:38:27,052::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:28,077::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:28,078::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3329::DEBUG::2014-10-15 06:38:28,079::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:28,080::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:28,080::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3330::DEBUG::2014-10-15 06:38:28,081::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3330::DEBUG::2014-10-15 06:38:28,082::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:38:28,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:28,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157974 s, 64.8 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:30,115::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:30,115::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:30,117::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:38:30,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:30,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0148402 s, 69.0 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:31,126::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:31,126::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3331::DEBUG::2014-10-15 06:38:31,127::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:31,129::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:31,129::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3332::DEBUG::2014-10-15 06:38:31,130::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3332::DEBUG::2014-10-15 06:38:31,130::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:38:32,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:32,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113246 s, 90.4 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:38:33,053::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:33,053::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:33,054::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:33,054::__init__::232::IOProcess::(_processLogs) (2459) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:38:33,054::__init__::232::IOProcess::(_processLogs) (2459) Queuing response >Thread-24::DEBUG::2014-10-15 06:38:33,054::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:38:33,060::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000393969 s, 1.2 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:38:33,060::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:33,061::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:33,061::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:33,061::__init__::232::IOProcess::(_processLogs) (2460) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:38:33,061::__init__::232::IOProcess::(_processLogs) (2460) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:33,061::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:33,062::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:33,062::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:33,062::__init__::232::IOProcess::(_processLogs) (2461) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:33,062::__init__::232::IOProcess::(_processLogs) (2461) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:33,062::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:33,062::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:33,062::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:33,062::__init__::232::IOProcess::(_processLogs) (2462) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:33,063::__init__::232::IOProcess::(_processLogs) (2462) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:33,063::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:33,063::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:33,063::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:33,063::__init__::232::IOProcess::(_processLogs) (2463) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:33,063::__init__::232::IOProcess::(_processLogs) (2463) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:34,180::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:34,180::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3333::DEBUG::2014-10-15 06:38:34,181::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:34,182::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:34,183::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3334::DEBUG::2014-10-15 06:38:34,184::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3334::DEBUG::2014-10-15 06:38:34,184::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:38:34,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:34,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116401 s, 88.0 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:35,123::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:35,123::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:35,124::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:38:36,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:36,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172303 s, 59.4 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:37,068::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:37,069::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3335::DEBUG::2014-10-15 06:38:37,069::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3335::DEBUG::2014-10-15 06:38:37,070::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb75cb86-cb56-4f7a-8508-9bcd01760d7d`::moving from state init -> state preparing >Thread-3335::INFO::2014-10-15 06:38:37,070::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3335::INFO::2014-10-15 06:38:37,082::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3335::DEBUG::2014-10-15 06:38:37,082::task::1191::Storage.TaskManager.Task::(prepare) Task=`eb75cb86-cb56-4f7a-8508-9bcd01760d7d`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3335::DEBUG::2014-10-15 06:38:37,082::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb75cb86-cb56-4f7a-8508-9bcd01760d7d`::moving from state preparing -> state finished >Thread-3335::DEBUG::2014-10-15 06:38:37,082::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3335::DEBUG::2014-10-15 06:38:37,082::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3335::DEBUG::2014-10-15 06:38:37,082::task::993::Storage.TaskManager.Task::(_decref) Task=`eb75cb86-cb56-4f7a-8508-9bcd01760d7d`::ref 0 aborting False >Thread-3335::DEBUG::2014-10-15 06:38:37,082::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3335::DEBUG::2014-10-15 06:38:37,083::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:37,089::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:37,089::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3336::DEBUG::2014-10-15 06:38:37,089::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3336::DEBUG::2014-10-15 06:38:37,090::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5fc041f-2b6b-41f4-81a8-e7b7c216cc6f`::moving from state init -> state preparing >Thread-3336::INFO::2014-10-15 06:38:37,090::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3336::DEBUG::2014-10-15 06:38:37,091::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`37da84cb-2530-4b23-b21e-0a9f4928f0c2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3336::DEBUG::2014-10-15 06:38:37,091::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3336::DEBUG::2014-10-15 06:38:37,091::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3336::DEBUG::2014-10-15 06:38:37,091::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`37da84cb-2530-4b23-b21e-0a9f4928f0c2`::Granted request >Thread-3336::DEBUG::2014-10-15 06:38:37,091::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d5fc041f-2b6b-41f4-81a8-e7b7c216cc6f`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3336::DEBUG::2014-10-15 06:38:37,091::task::993::Storage.TaskManager.Task::(_decref) Task=`d5fc041f-2b6b-41f4-81a8-e7b7c216cc6f`::ref 1 aborting False >Thread-3336::INFO::2014-10-15 06:38:37,109::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3336::DEBUG::2014-10-15 06:38:37,110::task::1191::Storage.TaskManager.Task::(prepare) Task=`d5fc041f-2b6b-41f4-81a8-e7b7c216cc6f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3336::DEBUG::2014-10-15 06:38:37,110::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5fc041f-2b6b-41f4-81a8-e7b7c216cc6f`::moving from state preparing -> state finished >Thread-3336::DEBUG::2014-10-15 06:38:37,110::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3336::DEBUG::2014-10-15 06:38:37,110::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3336::DEBUG::2014-10-15 06:38:37,110::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3336::DEBUG::2014-10-15 06:38:37,110::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3336::DEBUG::2014-10-15 06:38:37,110::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3336::DEBUG::2014-10-15 06:38:37,110::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3336::DEBUG::2014-10-15 06:38:37,110::task::993::Storage.TaskManager.Task::(_decref) Task=`d5fc041f-2b6b-41f4-81a8-e7b7c216cc6f`::ref 0 aborting False >Thread-3336::DEBUG::2014-10-15 06:38:37,111::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3336::DEBUG::2014-10-15 06:38:37,111::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:37,205::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:37,206::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3337::DEBUG::2014-10-15 06:38:37,207::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:37,208::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:37,209::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3338::DEBUG::2014-10-15 06:38:37,210::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3338::DEBUG::2014-10-15 06:38:37,210::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:38:37,319::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 47. >Dummy-838::DEBUG::2014-10-15 06:38:38,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:38,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114489 s, 89.4 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:40,125::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:40,126::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:40,126::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:40,302::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:40,303::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3339::DEBUG::2014-10-15 06:38:40,304::task::595::Storage.TaskManager.Task::(_updateState) Task=`744c1511-2bc3-4c8b-9269-0023cea19ab0`::moving from state init -> state preparing >Thread-3339::INFO::2014-10-15 06:38:40,304::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-3339::INFO::2014-10-15 06:38:40,304::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000393969', 'lastCheck': '7.2', 'valid': True}} >Thread-3339::DEBUG::2014-10-15 06:38:40,305::task::1191::Storage.TaskManager.Task::(prepare) Task=`744c1511-2bc3-4c8b-9269-0023cea19ab0`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000393969', 'lastCheck': '7.2', 'valid': True}} >Thread-3339::DEBUG::2014-10-15 06:38:40,305::task::595::Storage.TaskManager.Task::(_updateState) Task=`744c1511-2bc3-4c8b-9269-0023cea19ab0`::moving from state preparing -> state finished >Thread-3339::DEBUG::2014-10-15 06:38:40,305::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3339::DEBUG::2014-10-15 06:38:40,305::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3339::DEBUG::2014-10-15 06:38:40,305::task::993::Storage.TaskManager.Task::(_decref) Task=`744c1511-2bc3-4c8b-9269-0023cea19ab0`::ref 0 aborting False >Thread-3339::ERROR::2014-10-15 06:38:40,306::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-3339::DEBUG::2014-10-15 06:38:40,306::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:40,318::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:40,319::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3340::DEBUG::2014-10-15 06:38:40,320::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3340::DEBUG::2014-10-15 06:38:40,320::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:38:40,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:40,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0154902 s, 66.1 MB/s\n'; <rc> = 0 >Dummy-838::DEBUG::2014-10-15 06:38:42,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:42,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114344 s, 89.6 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:38:43,064::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:43,065::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:43,065::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:43,065::__init__::232::IOProcess::(_processLogs) (2464) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:38:43,065::__init__::232::IOProcess::(_processLogs) (2464) Queuing response >Thread-24::DEBUG::2014-10-15 06:38:43,065::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:38:43,071::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000395875 s, 1.2 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:38:43,072::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:43,072::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:43,072::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:43,072::__init__::232::IOProcess::(_processLogs) (2465) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:38:43,072::__init__::232::IOProcess::(_processLogs) (2465) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:43,073::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:43,073::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:43,073::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:43,073::__init__::232::IOProcess::(_processLogs) (2466) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:43,073::__init__::232::IOProcess::(_processLogs) (2466) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:43,073::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:43,074::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:43,074::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:43,074::__init__::232::IOProcess::(_processLogs) (2467) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:43,074::__init__::232::IOProcess::(_processLogs) (2467) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:43,074::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:43,074::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:43,074::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:43,074::__init__::232::IOProcess::(_processLogs) (2468) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:43,075::__init__::232::IOProcess::(_processLogs) (2468) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:43,440::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:43,441::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3341::DEBUG::2014-10-15 06:38:43,442::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:43,443::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:43,443::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3342::DEBUG::2014-10-15 06:38:43,444::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3342::DEBUG::2014-10-15 06:38:43,445::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:38:44,936::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:44,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114631 s, 89.3 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:45,131::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:45,131::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:45,132::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:46,479::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:46,479::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3343::DEBUG::2014-10-15 06:38:46,480::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:46,481::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:46,482::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3344::DEBUG::2014-10-15 06:38:46,483::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3344::DEBUG::2014-10-15 06:38:46,483::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:38:46,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:46,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114106 s, 89.7 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:47,129::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:47,130::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3345::DEBUG::2014-10-15 06:38:47,130::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3345::DEBUG::2014-10-15 06:38:47,131::task::595::Storage.TaskManager.Task::(_updateState) Task=`82236a8d-9780-4984-957f-835090767472`::moving from state init -> state preparing >Thread-3345::INFO::2014-10-15 06:38:47,131::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3345::INFO::2014-10-15 06:38:47,142::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3345::DEBUG::2014-10-15 06:38:47,143::task::1191::Storage.TaskManager.Task::(prepare) Task=`82236a8d-9780-4984-957f-835090767472`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3345::DEBUG::2014-10-15 06:38:47,143::task::595::Storage.TaskManager.Task::(_updateState) Task=`82236a8d-9780-4984-957f-835090767472`::moving from state preparing -> state finished >Thread-3345::DEBUG::2014-10-15 06:38:47,143::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3345::DEBUG::2014-10-15 06:38:47,143::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3345::DEBUG::2014-10-15 06:38:47,143::task::993::Storage.TaskManager.Task::(_decref) Task=`82236a8d-9780-4984-957f-835090767472`::ref 0 aborting False >Thread-3345::DEBUG::2014-10-15 06:38:47,143::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3345::DEBUG::2014-10-15 06:38:47,143::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:47,151::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:47,151::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3346::DEBUG::2014-10-15 06:38:47,151::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3346::DEBUG::2014-10-15 06:38:47,152::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bd93b68-fec6-408a-b53e-bc4ddaa7d83d`::moving from state init -> state preparing >Thread-3346::INFO::2014-10-15 06:38:47,152::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3346::DEBUG::2014-10-15 06:38:47,153::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`f6a4ebd5-67e0-4581-9ecc-5146cbd32d1b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3346::DEBUG::2014-10-15 06:38:47,153::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3346::DEBUG::2014-10-15 06:38:47,153::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3346::DEBUG::2014-10-15 06:38:47,153::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`f6a4ebd5-67e0-4581-9ecc-5146cbd32d1b`::Granted request >Thread-3346::DEBUG::2014-10-15 06:38:47,153::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8bd93b68-fec6-408a-b53e-bc4ddaa7d83d`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3346::DEBUG::2014-10-15 06:38:47,153::task::993::Storage.TaskManager.Task::(_decref) Task=`8bd93b68-fec6-408a-b53e-bc4ddaa7d83d`::ref 1 aborting False >Thread-3346::INFO::2014-10-15 06:38:47,171::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3346::DEBUG::2014-10-15 06:38:47,171::task::1191::Storage.TaskManager.Task::(prepare) Task=`8bd93b68-fec6-408a-b53e-bc4ddaa7d83d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3346::DEBUG::2014-10-15 06:38:47,171::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bd93b68-fec6-408a-b53e-bc4ddaa7d83d`::moving from state preparing -> state finished >Thread-3346::DEBUG::2014-10-15 06:38:47,171::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3346::DEBUG::2014-10-15 06:38:47,171::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3346::DEBUG::2014-10-15 06:38:47,171::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3346::DEBUG::2014-10-15 06:38:47,172::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3346::DEBUG::2014-10-15 06:38:47,172::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3346::DEBUG::2014-10-15 06:38:47,172::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3346::DEBUG::2014-10-15 06:38:47,172::task::993::Storage.TaskManager.Task::(_decref) Task=`8bd93b68-fec6-408a-b53e-bc4ddaa7d83d`::ref 0 aborting False >Thread-3346::DEBUG::2014-10-15 06:38:47,172::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3346::DEBUG::2014-10-15 06:38:47,172::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:38:48,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:49,004::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114047 s, 89.8 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:49,506::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:49,507::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3347::DEBUG::2014-10-15 06:38:49,508::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:49,509::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:49,510::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3348::DEBUG::2014-10-15 06:38:49,511::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3348::DEBUG::2014-10-15 06:38:49,511::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:50,137::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:50,137::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:50,138::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:38:51,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:51,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114451 s, 89.5 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:52,534::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:52,534::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3349::DEBUG::2014-10-15 06:38:52,536::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:52,537::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:52,537::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3350::DEBUG::2014-10-15 06:38:52,538::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3350::DEBUG::2014-10-15 06:38:52,538::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:38:53,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:53,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114055 s, 89.8 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:38:53,075::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:53,076::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:53,076::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:53,076::__init__::232::IOProcess::(_processLogs) (2469) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:38:53,076::__init__::232::IOProcess::(_processLogs) (2469) Queuing response >Thread-24::DEBUG::2014-10-15 06:38:53,076::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:38:53,082::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000393294 s, 1.2 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:38:53,082::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:53,083::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:53,083::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:53,083::__init__::232::IOProcess::(_processLogs) (2470) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:38:53,083::__init__::232::IOProcess::(_processLogs) (2470) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:53,083::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:53,084::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:53,084::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:53,084::__init__::232::IOProcess::(_processLogs) (2471) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:53,084::__init__::232::IOProcess::(_processLogs) (2471) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:53,084::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:53,084::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:53,084::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:53,084::__init__::232::IOProcess::(_processLogs) (2472) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:53,084::__init__::232::IOProcess::(_processLogs) (2472) Queuing response >Thread-21::DEBUG::2014-10-15 06:38:53,085::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:38:53,085::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:38:53,085::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:38:53,085::__init__::232::IOProcess::(_processLogs) (2473) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:38:53,085::__init__::232::IOProcess::(_processLogs) (2473) Queuing response >Dummy-838::DEBUG::2014-10-15 06:38:55,065::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:55,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180377 s, 56.8 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:55,143::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:55,144::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:38:55,144::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:55,563::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:55,564::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3351::DEBUG::2014-10-15 06:38:55,565::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f099264-ae55-4628-ab17-9b77cdc7ccdf`::moving from state init -> state preparing >Thread-3351::INFO::2014-10-15 06:38:55,565::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-3351::INFO::2014-10-15 06:38:55,565::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000393294', 'lastCheck': '2.5', 'valid': True}} >Thread-3351::DEBUG::2014-10-15 06:38:55,565::task::1191::Storage.TaskManager.Task::(prepare) Task=`0f099264-ae55-4628-ab17-9b77cdc7ccdf`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000393294', 'lastCheck': '2.5', 'valid': True}} >Thread-3351::DEBUG::2014-10-15 06:38:55,565::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f099264-ae55-4628-ab17-9b77cdc7ccdf`::moving from state preparing -> state finished >Thread-3351::DEBUG::2014-10-15 06:38:55,566::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3351::DEBUG::2014-10-15 06:38:55,566::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3351::DEBUG::2014-10-15 06:38:55,566::task::993::Storage.TaskManager.Task::(_decref) Task=`0f099264-ae55-4628-ab17-9b77cdc7ccdf`::ref 0 aborting False >Thread-3351::ERROR::2014-10-15 06:38:55,566::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-3351::DEBUG::2014-10-15 06:38:55,567::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:55,579::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:55,579::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3352::DEBUG::2014-10-15 06:38:55,580::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3352::DEBUG::2014-10-15 06:38:55,580::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:38:57,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:57,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114452 s, 89.5 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:57,211::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:57,213::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3353::DEBUG::2014-10-15 06:38:57,213::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3353::DEBUG::2014-10-15 06:38:57,215::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb9f64d1-323a-433a-a0a1-45dbffa23cad`::moving from state init -> state preparing >Thread-3353::INFO::2014-10-15 06:38:57,216::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3353::INFO::2014-10-15 06:38:57,228::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3353::DEBUG::2014-10-15 06:38:57,228::task::1191::Storage.TaskManager.Task::(prepare) Task=`fb9f64d1-323a-433a-a0a1-45dbffa23cad`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3353::DEBUG::2014-10-15 06:38:57,228::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb9f64d1-323a-433a-a0a1-45dbffa23cad`::moving from state preparing -> state finished >Thread-3353::DEBUG::2014-10-15 06:38:57,229::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3353::DEBUG::2014-10-15 06:38:57,229::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3353::DEBUG::2014-10-15 06:38:57,229::task::993::Storage.TaskManager.Task::(_decref) Task=`fb9f64d1-323a-433a-a0a1-45dbffa23cad`::ref 0 aborting False >Thread-3353::DEBUG::2014-10-15 06:38:57,229::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3353::DEBUG::2014-10-15 06:38:57,229::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:57,237::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:57,237::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3354::DEBUG::2014-10-15 06:38:57,237::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3354::DEBUG::2014-10-15 06:38:57,238::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a7d8485-870e-4dc8-adae-9b7a9f456b1d`::moving from state init -> state preparing >Thread-3354::INFO::2014-10-15 06:38:57,238::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3354::DEBUG::2014-10-15 06:38:57,239::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`b208a58b-8083-40d0-9502-cc00dcbce8c7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3354::DEBUG::2014-10-15 06:38:57,239::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3354::DEBUG::2014-10-15 06:38:57,239::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3354::DEBUG::2014-10-15 06:38:57,239::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`b208a58b-8083-40d0-9502-cc00dcbce8c7`::Granted request >Thread-3354::DEBUG::2014-10-15 06:38:57,239::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4a7d8485-870e-4dc8-adae-9b7a9f456b1d`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3354::DEBUG::2014-10-15 06:38:57,239::task::993::Storage.TaskManager.Task::(_decref) Task=`4a7d8485-870e-4dc8-adae-9b7a9f456b1d`::ref 1 aborting False >Thread-3354::INFO::2014-10-15 06:38:57,257::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3354::DEBUG::2014-10-15 06:38:57,257::task::1191::Storage.TaskManager.Task::(prepare) Task=`4a7d8485-870e-4dc8-adae-9b7a9f456b1d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3354::DEBUG::2014-10-15 06:38:57,257::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a7d8485-870e-4dc8-adae-9b7a9f456b1d`::moving from state preparing -> state finished >Thread-3354::DEBUG::2014-10-15 06:38:57,257::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3354::DEBUG::2014-10-15 06:38:57,258::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3354::DEBUG::2014-10-15 06:38:57,258::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3354::DEBUG::2014-10-15 06:38:57,258::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3354::DEBUG::2014-10-15 06:38:57,258::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3354::DEBUG::2014-10-15 06:38:57,258::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3354::DEBUG::2014-10-15 06:38:57,258::task::993::Storage.TaskManager.Task::(_decref) Task=`4a7d8485-870e-4dc8-adae-9b7a9f456b1d`::ref 0 aborting False >Thread-3354::DEBUG::2014-10-15 06:38:57,258::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3354::DEBUG::2014-10-15 06:38:57,258::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:58,695::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:58,695::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3355::DEBUG::2014-10-15 06:38:58,696::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:38:58,697::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:38:58,698::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3356::DEBUG::2014-10-15 06:38:58,699::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3356::DEBUG::2014-10-15 06:38:58,699::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:38:59,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:38:59,140::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113382 s, 90.3 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:00,150::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:00,151::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:00,151::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:39:01,148::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:01,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158976 s, 64.4 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:01,736::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:01,736::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3357::DEBUG::2014-10-15 06:39:01,737::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:01,739::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:01,739::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3358::DEBUG::2014-10-15 06:39:01,740::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3358::DEBUG::2014-10-15 06:39:01,741::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-21::DEBUG::2014-10-15 06:39:03,086::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:03,086::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:03,087::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:03,087::__init__::232::IOProcess::(_processLogs) (2474) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:39:03,087::__init__::232::IOProcess::(_processLogs) (2474) Queuing response >Thread-24::DEBUG::2014-10-15 06:39:03,087::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:39:03,093::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000378408 s, 1.2 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:39:03,093::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:03,093::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:03,094::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:03,094::__init__::232::IOProcess::(_processLogs) (2475) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:39:03,094::__init__::232::IOProcess::(_processLogs) (2475) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:03,094::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:03,094::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:03,094::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:03,095::__init__::232::IOProcess::(_processLogs) (2476) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:03,095::__init__::232::IOProcess::(_processLogs) (2476) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:03,095::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:03,095::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:03,095::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:03,095::__init__::232::IOProcess::(_processLogs) (2477) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:03,095::__init__::232::IOProcess::(_processLogs) (2477) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:03,096::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:03,096::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:03,096::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:03,096::__init__::232::IOProcess::(_processLogs) (2478) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:03,096::__init__::232::IOProcess::(_processLogs) (2478) Queuing response >Dummy-838::DEBUG::2014-10-15 06:39:03,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:03,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114599 s, 89.4 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:04,762::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:04,762::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3359::DEBUG::2014-10-15 06:39:04,764::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:04,765::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:04,765::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3360::DEBUG::2014-10-15 06:39:04,766::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3360::DEBUG::2014-10-15 06:39:04,766::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-167::DEBUG::2014-10-15 06:39:05,128::task::595::Storage.TaskManager.Task::(_updateState) Task=`02a0a7bb-13bb-4cbf-8796-0254a10a357d`::moving from state init -> state preparing >Thread-167::INFO::2014-10-15 06:39:05,128::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-21::DEBUG::2014-10-15 06:39:05,129::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:05,129::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:05,129::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:05,129::__init__::232::IOProcess::(_processLogs) (2479) Got request for method 'stat' >Thread-21::DEBUG::2014-10-15 06:39:05,130::__init__::232::IOProcess::(_processLogs) (2479) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:05,131::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:05,131::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:05,131::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:05,131::__init__::232::IOProcess::(_processLogs) (2480) Got request for method 'stat' >Thread-21::DEBUG::2014-10-15 06:39:05,131::__init__::232::IOProcess::(_processLogs) (2480) Queuing response >Thread-167::INFO::2014-10-15 06:39:05,131::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-167::DEBUG::2014-10-15 06:39:05,131::task::1191::Storage.TaskManager.Task::(prepare) Task=`02a0a7bb-13bb-4cbf-8796-0254a10a357d`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-167::DEBUG::2014-10-15 06:39:05,131::task::595::Storage.TaskManager.Task::(_updateState) Task=`02a0a7bb-13bb-4cbf-8796-0254a10a357d`::moving from state preparing -> state finished >Thread-167::DEBUG::2014-10-15 06:39:05,131::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-167::DEBUG::2014-10-15 06:39:05,132::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-167::DEBUG::2014-10-15 06:39:05,132::task::993::Storage.TaskManager.Task::(_decref) Task=`02a0a7bb-13bb-4cbf-8796-0254a10a357d`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:05,157::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:05,157::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:05,157::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:39:05,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:05,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115016 s, 89.0 MB/s\n'; <rc> = 0 >Dummy-838::DEBUG::2014-10-15 06:39:07,230::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:07,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178259 s, 57.4 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:07,284::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:07,285::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3361::DEBUG::2014-10-15 06:39:07,285::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3361::DEBUG::2014-10-15 06:39:07,288::task::595::Storage.TaskManager.Task::(_updateState) Task=`f83b30e9-f2b2-412b-a3f2-c682941eb4a7`::moving from state init -> state preparing >Thread-3361::INFO::2014-10-15 06:39:07,288::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3361::INFO::2014-10-15 06:39:07,300::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3361::DEBUG::2014-10-15 06:39:07,300::task::1191::Storage.TaskManager.Task::(prepare) Task=`f83b30e9-f2b2-412b-a3f2-c682941eb4a7`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3361::DEBUG::2014-10-15 06:39:07,300::task::595::Storage.TaskManager.Task::(_updateState) Task=`f83b30e9-f2b2-412b-a3f2-c682941eb4a7`::moving from state preparing -> state finished >Thread-3361::DEBUG::2014-10-15 06:39:07,300::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3361::DEBUG::2014-10-15 06:39:07,300::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3361::DEBUG::2014-10-15 06:39:07,300::task::993::Storage.TaskManager.Task::(_decref) Task=`f83b30e9-f2b2-412b-a3f2-c682941eb4a7`::ref 0 aborting False >Thread-3361::DEBUG::2014-10-15 06:39:07,301::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3361::DEBUG::2014-10-15 06:39:07,301::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:07,309::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-3362::DEBUG::2014-10-15 06:39:07,309::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >JsonRpcServer::DEBUG::2014-10-15 06:39:07,309::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3362::DEBUG::2014-10-15 06:39:07,310::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea5319b7-3218-4df5-9c0d-5354d2c3cdfd`::moving from state init -> state preparing >Thread-3362::INFO::2014-10-15 06:39:07,310::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3362::DEBUG::2014-10-15 06:39:07,311::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`c5fa17d9-dc81-47a0-9f45-d7c2e35e4243`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3362::DEBUG::2014-10-15 06:39:07,311::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3362::DEBUG::2014-10-15 06:39:07,311::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3362::DEBUG::2014-10-15 06:39:07,311::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`c5fa17d9-dc81-47a0-9f45-d7c2e35e4243`::Granted request >Thread-3362::DEBUG::2014-10-15 06:39:07,311::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ea5319b7-3218-4df5-9c0d-5354d2c3cdfd`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3362::DEBUG::2014-10-15 06:39:07,311::task::993::Storage.TaskManager.Task::(_decref) Task=`ea5319b7-3218-4df5-9c0d-5354d2c3cdfd`::ref 1 aborting False >Thread-3362::INFO::2014-10-15 06:39:07,329::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3362::DEBUG::2014-10-15 06:39:07,329::task::1191::Storage.TaskManager.Task::(prepare) Task=`ea5319b7-3218-4df5-9c0d-5354d2c3cdfd`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3362::DEBUG::2014-10-15 06:39:07,329::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea5319b7-3218-4df5-9c0d-5354d2c3cdfd`::moving from state preparing -> state finished >Thread-3362::DEBUG::2014-10-15 06:39:07,330::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3362::DEBUG::2014-10-15 06:39:07,330::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3362::DEBUG::2014-10-15 06:39:07,330::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3362::DEBUG::2014-10-15 06:39:07,330::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3362::DEBUG::2014-10-15 06:39:07,330::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3362::DEBUG::2014-10-15 06:39:07,330::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3362::DEBUG::2014-10-15 06:39:07,330::task::993::Storage.TaskManager.Task::(_decref) Task=`ea5319b7-3218-4df5-9c0d-5354d2c3cdfd`::ref 0 aborting False >Thread-3362::DEBUG::2014-10-15 06:39:07,330::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3362::DEBUG::2014-10-15 06:39:07,331::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:39:07,352::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 47. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:07,789::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:07,789::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3363::DEBUG::2014-10-15 06:39:07,790::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:07,791::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:07,792::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3364::DEBUG::2014-10-15 06:39:07,793::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3364::DEBUG::2014-10-15 06:39:07,793::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:39:09,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:09,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113519 s, 90.2 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:10,163::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:10,164::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:10,164::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:10,811::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:10,811::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3365::DEBUG::2014-10-15 06:39:10,813::task::595::Storage.TaskManager.Task::(_updateState) Task=`d96aae0b-1282-42f2-bee4-09a9b1ca6987`::moving from state init -> state preparing >Thread-3365::INFO::2014-10-15 06:39:10,813::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-3365::INFO::2014-10-15 06:39:10,813::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000378408', 'lastCheck': '7.7', 'valid': True}} >Thread-3365::DEBUG::2014-10-15 06:39:10,813::task::1191::Storage.TaskManager.Task::(prepare) Task=`d96aae0b-1282-42f2-bee4-09a9b1ca6987`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000378408', 'lastCheck': '7.7', 'valid': True}} >Thread-3365::DEBUG::2014-10-15 06:39:10,813::task::595::Storage.TaskManager.Task::(_updateState) Task=`d96aae0b-1282-42f2-bee4-09a9b1ca6987`::moving from state preparing -> state finished >Thread-3365::DEBUG::2014-10-15 06:39:10,813::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3365::DEBUG::2014-10-15 06:39:10,813::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3365::DEBUG::2014-10-15 06:39:10,813::task::993::Storage.TaskManager.Task::(_decref) Task=`d96aae0b-1282-42f2-bee4-09a9b1ca6987`::ref 0 aborting False >Thread-3365::ERROR::2014-10-15 06:39:10,814::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-3365::DEBUG::2014-10-15 06:39:10,815::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:10,827::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:10,828::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3366::DEBUG::2014-10-15 06:39:10,829::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3366::DEBUG::2014-10-15 06:39:10,829::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:39:11,318::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:11,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113279 s, 90.4 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:39:13,097::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:13,097::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:13,097::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:13,098::__init__::232::IOProcess::(_processLogs) (2481) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:39:13,098::__init__::232::IOProcess::(_processLogs) (2481) Queuing response >Thread-24::DEBUG::2014-10-15 06:39:13,098::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:39:13,104::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000414126 s, 1.1 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:39:13,104::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:13,104::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:13,105::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:13,105::__init__::232::IOProcess::(_processLogs) (2482) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:39:13,105::__init__::232::IOProcess::(_processLogs) (2482) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:13,105::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:13,105::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:13,106::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:13,106::__init__::232::IOProcess::(_processLogs) (2483) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:13,106::__init__::232::IOProcess::(_processLogs) (2483) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:13,106::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:13,106::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:13,106::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:13,106::__init__::232::IOProcess::(_processLogs) (2484) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:13,107::__init__::232::IOProcess::(_processLogs) (2484) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:13,107::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:13,107::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:13,107::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:13,107::__init__::232::IOProcess::(_processLogs) (2485) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:13,107::__init__::232::IOProcess::(_processLogs) (2485) Queuing response >Dummy-838::DEBUG::2014-10-15 06:39:13,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:13,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017914 s, 57.2 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:13,966::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:13,967::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3367::DEBUG::2014-10-15 06:39:13,970::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:13,972::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:13,973::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3368::DEBUG::2014-10-15 06:39:13,975::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3368::DEBUG::2014-10-15 06:39:13,977::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:15,170::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:15,170::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:15,171::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:39:15,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:15,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166296 s, 61.6 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:17,011::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:17,012::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3369::DEBUG::2014-10-15 06:39:17,013::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:17,014::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:17,014::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3370::DEBUG::2014-10-15 06:39:17,015::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3370::DEBUG::2014-10-15 06:39:17,016::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:17,401::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:17,401::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3371::DEBUG::2014-10-15 06:39:17,402::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3371::DEBUG::2014-10-15 06:39:17,403::task::595::Storage.TaskManager.Task::(_updateState) Task=`7c8dd529-5d15-4345-9c9c-b60a933962c0`::moving from state init -> state preparing >Thread-3371::INFO::2014-10-15 06:39:17,403::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Dummy-838::DEBUG::2014-10-15 06:39:17,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-3371::INFO::2014-10-15 06:39:17,415::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3371::DEBUG::2014-10-15 06:39:17,415::task::1191::Storage.TaskManager.Task::(prepare) Task=`7c8dd529-5d15-4345-9c9c-b60a933962c0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3371::DEBUG::2014-10-15 06:39:17,416::task::595::Storage.TaskManager.Task::(_updateState) Task=`7c8dd529-5d15-4345-9c9c-b60a933962c0`::moving from state preparing -> state finished >Thread-3371::DEBUG::2014-10-15 06:39:17,416::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3371::DEBUG::2014-10-15 06:39:17,416::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3371::DEBUG::2014-10-15 06:39:17,416::task::993::Storage.TaskManager.Task::(_decref) Task=`7c8dd529-5d15-4345-9c9c-b60a933962c0`::ref 0 aborting False >Thread-3371::DEBUG::2014-10-15 06:39:17,416::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3371::DEBUG::2014-10-15 06:39:17,417::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:17,425::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:17,427::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3372::DEBUG::2014-10-15 06:39:17,427::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3372::DEBUG::2014-10-15 06:39:17,429::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb068d99-48eb-40e1-9081-cdf488fa67b1`::moving from state init -> state preparing >Thread-3372::INFO::2014-10-15 06:39:17,430::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3372::DEBUG::2014-10-15 06:39:17,431::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`5049ac03-bd2e-4d26-ad2e-96d8b2eb0e23`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3372::DEBUG::2014-10-15 06:39:17,431::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3372::DEBUG::2014-10-15 06:39:17,432::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3372::DEBUG::2014-10-15 06:39:17,432::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`5049ac03-bd2e-4d26-ad2e-96d8b2eb0e23`::Granted request >Thread-3372::DEBUG::2014-10-15 06:39:17,434::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fb068d99-48eb-40e1-9081-cdf488fa67b1`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Dummy-838::DEBUG::2014-10-15 06:39:17,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178719 s, 57.3 MB/s\n'; <rc> = 0 >Thread-3372::DEBUG::2014-10-15 06:39:17,434::task::993::Storage.TaskManager.Task::(_decref) Task=`fb068d99-48eb-40e1-9081-cdf488fa67b1`::ref 1 aborting False >Thread-3372::INFO::2014-10-15 06:39:17,470::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3372::DEBUG::2014-10-15 06:39:17,470::task::1191::Storage.TaskManager.Task::(prepare) Task=`fb068d99-48eb-40e1-9081-cdf488fa67b1`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3372::DEBUG::2014-10-15 06:39:17,470::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb068d99-48eb-40e1-9081-cdf488fa67b1`::moving from state preparing -> state finished >Thread-3372::DEBUG::2014-10-15 06:39:17,470::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3372::DEBUG::2014-10-15 06:39:17,470::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3372::DEBUG::2014-10-15 06:39:17,470::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3372::DEBUG::2014-10-15 06:39:17,470::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3372::DEBUG::2014-10-15 06:39:17,471::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3372::DEBUG::2014-10-15 06:39:17,471::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3372::DEBUG::2014-10-15 06:39:17,471::task::993::Storage.TaskManager.Task::(_decref) Task=`fb068d99-48eb-40e1-9081-cdf488fa67b1`::ref 0 aborting False >Thread-3372::DEBUG::2014-10-15 06:39:17,471::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3372::DEBUG::2014-10-15 06:39:17,471::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:39:19,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:19,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114521 s, 89.4 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:20,037::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:20,038::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3373::DEBUG::2014-10-15 06:39:20,039::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:20,040::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:20,041::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3374::DEBUG::2014-10-15 06:39:20,042::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3374::DEBUG::2014-10-15 06:39:20,042::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:20,176::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:20,177::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:20,177::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:39:21,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:21,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116235 s, 88.1 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:23,064::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:23,065::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3375::DEBUG::2014-10-15 06:39:23,066::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:23,067::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:23,067::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3376::DEBUG::2014-10-15 06:39:23,068::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3376::DEBUG::2014-10-15 06:39:23,068::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-21::DEBUG::2014-10-15 06:39:23,108::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:23,108::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:23,108::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:23,109::__init__::232::IOProcess::(_processLogs) (2486) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:39:23,109::__init__::232::IOProcess::(_processLogs) (2486) Queuing response >Thread-24::DEBUG::2014-10-15 06:39:23,109::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:39:23,115::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000453074 s, 1.0 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:39:23,116::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:23,116::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:23,116::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:23,116::__init__::232::IOProcess::(_processLogs) (2487) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:39:23,116::__init__::232::IOProcess::(_processLogs) (2487) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:23,117::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:23,117::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:23,117::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:23,117::__init__::232::IOProcess::(_processLogs) (2488) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:23,117::__init__::232::IOProcess::(_processLogs) (2488) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:23,117::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:23,118::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:23,118::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:23,118::__init__::232::IOProcess::(_processLogs) (2489) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:23,118::__init__::232::IOProcess::(_processLogs) (2489) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:23,118::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:23,118::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:23,118::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:23,118::__init__::232::IOProcess::(_processLogs) (2490) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:23,119::__init__::232::IOProcess::(_processLogs) (2490) Queuing response >Dummy-838::DEBUG::2014-10-15 06:39:23,517::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:23,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0146879 s, 69.7 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:25,183::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:25,184::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:25,184::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:39:25,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:25,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163195 s, 62.7 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:26,086::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:26,087::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3377::DEBUG::2014-10-15 06:39:26,088::task::595::Storage.TaskManager.Task::(_updateState) Task=`6423fcbb-2fc6-452e-b505-f8366a6a5788`::moving from state init -> state preparing >Thread-3377::INFO::2014-10-15 06:39:26,088::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-3377::INFO::2014-10-15 06:39:26,089::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000453074', 'lastCheck': '3.0', 'valid': True}} >Thread-3377::DEBUG::2014-10-15 06:39:26,089::task::1191::Storage.TaskManager.Task::(prepare) Task=`6423fcbb-2fc6-452e-b505-f8366a6a5788`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000453074', 'lastCheck': '3.0', 'valid': True}} >Thread-3377::DEBUG::2014-10-15 06:39:26,089::task::595::Storage.TaskManager.Task::(_updateState) Task=`6423fcbb-2fc6-452e-b505-f8366a6a5788`::moving from state preparing -> state finished >Thread-3377::DEBUG::2014-10-15 06:39:26,089::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3377::DEBUG::2014-10-15 06:39:26,089::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3377::DEBUG::2014-10-15 06:39:26,089::task::993::Storage.TaskManager.Task::(_decref) Task=`6423fcbb-2fc6-452e-b505-f8366a6a5788`::ref 0 aborting False >Thread-3377::ERROR::2014-10-15 06:39:26,090::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-3377::DEBUG::2014-10-15 06:39:26,090::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:26,097::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:26,098::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3378::DEBUG::2014-10-15 06:39:26,099::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3378::DEBUG::2014-10-15 06:39:26,099::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:27,495::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:27,496::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3379::DEBUG::2014-10-15 06:39:27,496::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3379::DEBUG::2014-10-15 06:39:27,497::task::595::Storage.TaskManager.Task::(_updateState) Task=`65e17074-9dbe-40f3-a594-71a95aa91153`::moving from state init -> state preparing >Thread-3379::INFO::2014-10-15 06:39:27,497::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3379::INFO::2014-10-15 06:39:27,510::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3379::DEBUG::2014-10-15 06:39:27,510::task::1191::Storage.TaskManager.Task::(prepare) Task=`65e17074-9dbe-40f3-a594-71a95aa91153`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3379::DEBUG::2014-10-15 06:39:27,510::task::595::Storage.TaskManager.Task::(_updateState) Task=`65e17074-9dbe-40f3-a594-71a95aa91153`::moving from state preparing -> state finished >Thread-3379::DEBUG::2014-10-15 06:39:27,510::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3379::DEBUG::2014-10-15 06:39:27,510::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3379::DEBUG::2014-10-15 06:39:27,510::task::993::Storage.TaskManager.Task::(_decref) Task=`65e17074-9dbe-40f3-a594-71a95aa91153`::ref 0 aborting False >Thread-3379::DEBUG::2014-10-15 06:39:27,511::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3379::DEBUG::2014-10-15 06:39:27,511::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:27,518::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:27,519::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3380::DEBUG::2014-10-15 06:39:27,519::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3380::DEBUG::2014-10-15 06:39:27,520::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f227988-d17a-438c-b4ed-078eabaac201`::moving from state init -> state preparing >Thread-3380::INFO::2014-10-15 06:39:27,520::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3380::DEBUG::2014-10-15 06:39:27,520::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`96593d87-f492-4121-bfe8-4af2074e311b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3380::DEBUG::2014-10-15 06:39:27,520::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3380::DEBUG::2014-10-15 06:39:27,520::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3380::DEBUG::2014-10-15 06:39:27,520::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`96593d87-f492-4121-bfe8-4af2074e311b`::Granted request >Thread-3380::DEBUG::2014-10-15 06:39:27,521::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5f227988-d17a-438c-b4ed-078eabaac201`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3380::DEBUG::2014-10-15 06:39:27,521::task::993::Storage.TaskManager.Task::(_decref) Task=`5f227988-d17a-438c-b4ed-078eabaac201`::ref 1 aborting False >Thread-3380::INFO::2014-10-15 06:39:27,539::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3380::DEBUG::2014-10-15 06:39:27,539::task::1191::Storage.TaskManager.Task::(prepare) Task=`5f227988-d17a-438c-b4ed-078eabaac201`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3380::DEBUG::2014-10-15 06:39:27,539::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f227988-d17a-438c-b4ed-078eabaac201`::moving from state preparing -> state finished >Thread-3380::DEBUG::2014-10-15 06:39:27,539::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3380::DEBUG::2014-10-15 06:39:27,539::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3380::DEBUG::2014-10-15 06:39:27,539::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3380::DEBUG::2014-10-15 06:39:27,540::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3380::DEBUG::2014-10-15 06:39:27,540::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3380::DEBUG::2014-10-15 06:39:27,540::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3380::DEBUG::2014-10-15 06:39:27,540::task::993::Storage.TaskManager.Task::(_decref) Task=`5f227988-d17a-438c-b4ed-078eabaac201`::ref 0 aborting False >Thread-3380::DEBUG::2014-10-15 06:39:27,540::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3380::DEBUG::2014-10-15 06:39:27,540::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:39:27,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:27,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0107469 s, 95.3 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:29,246::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:29,246::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3381::DEBUG::2014-10-15 06:39:29,248::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:29,248::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:29,249::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3382::DEBUG::2014-10-15 06:39:29,250::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3382::DEBUG::2014-10-15 06:39:29,250::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:39:29,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:29,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121798 s, 84.1 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:30,190::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:30,190::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:30,191::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:39:31,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:31,650::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180095 s, 56.9 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:32,323::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:32,324::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3383::DEBUG::2014-10-15 06:39:32,325::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:32,327::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:32,327::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3384::DEBUG::2014-10-15 06:39:32,328::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3384::DEBUG::2014-10-15 06:39:32,328::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-21::DEBUG::2014-10-15 06:39:33,120::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:33,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:33,120::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:33,120::__init__::232::IOProcess::(_processLogs) (2491) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:39:33,120::__init__::232::IOProcess::(_processLogs) (2491) Queuing response >Thread-24::DEBUG::2014-10-15 06:39:33,120::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:39:33,126::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000411865 s, 1.1 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:39:33,126::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:33,127::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:33,127::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:33,127::__init__::232::IOProcess::(_processLogs) (2492) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:39:33,127::__init__::232::IOProcess::(_processLogs) (2492) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:33,127::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:33,128::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:33,128::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:33,128::__init__::232::IOProcess::(_processLogs) (2493) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:33,128::__init__::232::IOProcess::(_processLogs) (2493) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:33,128::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:33,128::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:33,128::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:33,129::__init__::232::IOProcess::(_processLogs) (2494) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:33,129::__init__::232::IOProcess::(_processLogs) (2494) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:33,129::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:33,129::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:33,129::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:33,129::__init__::232::IOProcess::(_processLogs) (2495) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:33,129::__init__::232::IOProcess::(_processLogs) (2495) Queuing response >Dummy-838::DEBUG::2014-10-15 06:39:33,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:33,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0144046 s, 71.1 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:35,196::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:35,197::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:35,197::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:35,347::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:35,348::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3385::DEBUG::2014-10-15 06:39:35,349::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:35,350::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:35,351::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3386::DEBUG::2014-10-15 06:39:35,352::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3386::DEBUG::2014-10-15 06:39:35,352::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:39:35,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:35,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0107168 s, 95.6 MB/s\n'; <rc> = 0 >VM Channels Listener::DEBUG::2014-10-15 06:39:37,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 47. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:37,557::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:37,558::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3387::DEBUG::2014-10-15 06:39:37,558::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3387::DEBUG::2014-10-15 06:39:37,559::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce8c599a-f9db-42da-8cae-8deca3b2b1f2`::moving from state init -> state preparing >Thread-3387::INFO::2014-10-15 06:39:37,559::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3387::INFO::2014-10-15 06:39:37,572::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3387::DEBUG::2014-10-15 06:39:37,572::task::1191::Storage.TaskManager.Task::(prepare) Task=`ce8c599a-f9db-42da-8cae-8deca3b2b1f2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3387::DEBUG::2014-10-15 06:39:37,572::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce8c599a-f9db-42da-8cae-8deca3b2b1f2`::moving from state preparing -> state finished >Thread-3387::DEBUG::2014-10-15 06:39:37,572::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3387::DEBUG::2014-10-15 06:39:37,572::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3387::DEBUG::2014-10-15 06:39:37,572::task::993::Storage.TaskManager.Task::(_decref) Task=`ce8c599a-f9db-42da-8cae-8deca3b2b1f2`::ref 0 aborting False >Thread-3387::DEBUG::2014-10-15 06:39:37,572::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3387::DEBUG::2014-10-15 06:39:37,572::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:37,579::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:37,580::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3388::DEBUG::2014-10-15 06:39:37,580::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3388::DEBUG::2014-10-15 06:39:37,580::task::595::Storage.TaskManager.Task::(_updateState) Task=`e40b5725-409f-4b62-960e-688cb4cfced2`::moving from state init -> state preparing >Thread-3388::INFO::2014-10-15 06:39:37,581::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3388::DEBUG::2014-10-15 06:39:37,581::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`7a1af503-c2c7-4961-b40d-c4c1eb3edd29`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3388::DEBUG::2014-10-15 06:39:37,581::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3388::DEBUG::2014-10-15 06:39:37,581::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3388::DEBUG::2014-10-15 06:39:37,581::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`7a1af503-c2c7-4961-b40d-c4c1eb3edd29`::Granted request >Thread-3388::DEBUG::2014-10-15 06:39:37,582::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e40b5725-409f-4b62-960e-688cb4cfced2`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3388::DEBUG::2014-10-15 06:39:37,582::task::993::Storage.TaskManager.Task::(_decref) Task=`e40b5725-409f-4b62-960e-688cb4cfced2`::ref 1 aborting False >Thread-3388::INFO::2014-10-15 06:39:37,600::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3388::DEBUG::2014-10-15 06:39:37,600::task::1191::Storage.TaskManager.Task::(prepare) Task=`e40b5725-409f-4b62-960e-688cb4cfced2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3388::DEBUG::2014-10-15 06:39:37,600::task::595::Storage.TaskManager.Task::(_updateState) Task=`e40b5725-409f-4b62-960e-688cb4cfced2`::moving from state preparing -> state finished >Thread-3388::DEBUG::2014-10-15 06:39:37,600::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3388::DEBUG::2014-10-15 06:39:37,600::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3388::DEBUG::2014-10-15 06:39:37,600::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3388::DEBUG::2014-10-15 06:39:37,600::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3388::DEBUG::2014-10-15 06:39:37,600::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3388::DEBUG::2014-10-15 06:39:37,601::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3388::DEBUG::2014-10-15 06:39:37,601::task::993::Storage.TaskManager.Task::(_decref) Task=`e40b5725-409f-4b62-960e-688cb4cfced2`::ref 0 aborting False >Thread-3388::DEBUG::2014-10-15 06:39:37,601::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3388::DEBUG::2014-10-15 06:39:37,601::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:39:37,712::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:37,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0106132 s, 96.5 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:38,385::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:38,385::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3389::DEBUG::2014-10-15 06:39:38,386::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:38,388::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:38,388::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3390::DEBUG::2014-10-15 06:39:38,389::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3390::DEBUG::2014-10-15 06:39:38,389::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:39:39,736::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:39,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0150737 s, 67.9 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:40,203::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:40,204::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:40,204::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:41,409::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:41,410::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3391::DEBUG::2014-10-15 06:39:41,411::task::595::Storage.TaskManager.Task::(_updateState) Task=`da5ad590-a303-4bb1-ae92-6d2183697597`::moving from state init -> state preparing >Thread-3391::INFO::2014-10-15 06:39:41,411::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-3391::INFO::2014-10-15 06:39:41,411::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411865', 'lastCheck': '8.3', 'valid': True}} >Thread-3391::DEBUG::2014-10-15 06:39:41,411::task::1191::Storage.TaskManager.Task::(prepare) Task=`da5ad590-a303-4bb1-ae92-6d2183697597`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411865', 'lastCheck': '8.3', 'valid': True}} >Thread-3391::DEBUG::2014-10-15 06:39:41,411::task::595::Storage.TaskManager.Task::(_updateState) Task=`da5ad590-a303-4bb1-ae92-6d2183697597`::moving from state preparing -> state finished >Thread-3391::DEBUG::2014-10-15 06:39:41,412::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3391::DEBUG::2014-10-15 06:39:41,412::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3391::DEBUG::2014-10-15 06:39:41,412::task::993::Storage.TaskManager.Task::(_decref) Task=`da5ad590-a303-4bb1-ae92-6d2183697597`::ref 0 aborting False >Thread-3391::ERROR::2014-10-15 06:39:41,412::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-3391::DEBUG::2014-10-15 06:39:41,413::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:41,421::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:41,421::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3392::DEBUG::2014-10-15 06:39:41,422::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3392::DEBUG::2014-10-15 06:39:41,422::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:39:41,765::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:41,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114696 s, 89.3 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:39:43,131::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:43,131::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:43,131::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:43,131::__init__::232::IOProcess::(_processLogs) (2496) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:39:43,131::__init__::232::IOProcess::(_processLogs) (2496) Queuing response >Thread-24::DEBUG::2014-10-15 06:39:43,131::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:39:43,138::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00063874 s, 719 kB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:39:43,138::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:43,138::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:43,138::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:43,139::__init__::232::IOProcess::(_processLogs) (2497) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:39:43,139::__init__::232::IOProcess::(_processLogs) (2497) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:43,139::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:43,139::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:43,139::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:43,140::__init__::232::IOProcess::(_processLogs) (2498) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:43,140::__init__::232::IOProcess::(_processLogs) (2498) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:43,140::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:43,140::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:43,140::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:43,140::__init__::232::IOProcess::(_processLogs) (2499) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:43,140::__init__::232::IOProcess::(_processLogs) (2499) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:43,141::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:43,141::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:43,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:43,141::__init__::232::IOProcess::(_processLogs) (2500) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:43,141::__init__::232::IOProcess::(_processLogs) (2500) Queuing response >Dummy-838::DEBUG::2014-10-15 06:39:43,791::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:43,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169542 s, 60.4 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:44,525::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:44,527::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3393::DEBUG::2014-10-15 06:39:44,529::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:44,532::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:44,533::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3394::DEBUG::2014-10-15 06:39:44,535::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3394::DEBUG::2014-10-15 06:39:44,536::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:45,210::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:45,210::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:45,211::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:39:45,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:45,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115306 s, 88.8 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:47,571::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:47,572::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3395::DEBUG::2014-10-15 06:39:47,573::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:47,574::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:47,575::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3396::DEBUG::2014-10-15 06:39:47,576::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3396::DEBUG::2014-10-15 06:39:47,576::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:47,635::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-3397::DEBUG::2014-10-15 06:39:47,636::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >JsonRpcServer::DEBUG::2014-10-15 06:39:47,636::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3397::DEBUG::2014-10-15 06:39:47,637::task::595::Storage.TaskManager.Task::(_updateState) Task=`b036fe81-ec2f-44e3-8628-8eaf5dc4f5f2`::moving from state init -> state preparing >Thread-3397::INFO::2014-10-15 06:39:47,637::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3397::INFO::2014-10-15 06:39:47,649::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3397::DEBUG::2014-10-15 06:39:47,649::task::1191::Storage.TaskManager.Task::(prepare) Task=`b036fe81-ec2f-44e3-8628-8eaf5dc4f5f2`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3397::DEBUG::2014-10-15 06:39:47,649::task::595::Storage.TaskManager.Task::(_updateState) Task=`b036fe81-ec2f-44e3-8628-8eaf5dc4f5f2`::moving from state preparing -> state finished >Thread-3397::DEBUG::2014-10-15 06:39:47,649::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3397::DEBUG::2014-10-15 06:39:47,650::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3397::DEBUG::2014-10-15 06:39:47,650::task::993::Storage.TaskManager.Task::(_decref) Task=`b036fe81-ec2f-44e3-8628-8eaf5dc4f5f2`::ref 0 aborting False >Thread-3397::DEBUG::2014-10-15 06:39:47,650::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3397::DEBUG::2014-10-15 06:39:47,650::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:47,658::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:47,658::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3398::DEBUG::2014-10-15 06:39:47,658::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3398::DEBUG::2014-10-15 06:39:47,659::task::595::Storage.TaskManager.Task::(_updateState) Task=`83f2da74-969f-49d6-8b65-ad36937cc6a2`::moving from state init -> state preparing >Thread-3398::INFO::2014-10-15 06:39:47,659::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3398::DEBUG::2014-10-15 06:39:47,660::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`b4d7e623-10a2-4760-8d66-eae26f20fc71`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3398::DEBUG::2014-10-15 06:39:47,660::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3398::DEBUG::2014-10-15 06:39:47,660::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3398::DEBUG::2014-10-15 06:39:47,660::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`b4d7e623-10a2-4760-8d66-eae26f20fc71`::Granted request >Thread-3398::DEBUG::2014-10-15 06:39:47,660::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`83f2da74-969f-49d6-8b65-ad36937cc6a2`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3398::DEBUG::2014-10-15 06:39:47,660::task::993::Storage.TaskManager.Task::(_decref) Task=`83f2da74-969f-49d6-8b65-ad36937cc6a2`::ref 1 aborting False >Thread-3398::INFO::2014-10-15 06:39:47,678::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3398::DEBUG::2014-10-15 06:39:47,678::task::1191::Storage.TaskManager.Task::(prepare) Task=`83f2da74-969f-49d6-8b65-ad36937cc6a2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3398::DEBUG::2014-10-15 06:39:47,678::task::595::Storage.TaskManager.Task::(_updateState) Task=`83f2da74-969f-49d6-8b65-ad36937cc6a2`::moving from state preparing -> state finished >Thread-3398::DEBUG::2014-10-15 06:39:47,678::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3398::DEBUG::2014-10-15 06:39:47,678::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3398::DEBUG::2014-10-15 06:39:47,678::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3398::DEBUG::2014-10-15 06:39:47,679::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3398::DEBUG::2014-10-15 06:39:47,679::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3398::DEBUG::2014-10-15 06:39:47,679::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3398::DEBUG::2014-10-15 06:39:47,679::task::993::Storage.TaskManager.Task::(_decref) Task=`83f2da74-969f-49d6-8b65-ad36937cc6a2`::ref 0 aborting False >Thread-3398::DEBUG::2014-10-15 06:39:47,679::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3398::DEBUG::2014-10-15 06:39:47,679::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:39:47,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:47,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0106983 s, 95.7 MB/s\n'; <rc> = 0 >Dummy-838::DEBUG::2014-10-15 06:39:49,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:49,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114431 s, 89.5 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:50,216::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:50,217::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:50,217::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:50,599::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:50,599::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3399::DEBUG::2014-10-15 06:39:50,600::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:50,601::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:50,602::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3400::DEBUG::2014-10-15 06:39:50,603::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3400::DEBUG::2014-10-15 06:39:50,603::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:39:51,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:51,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011739 s, 87.2 MB/s\n'; <rc> = 0 >Thread-24::DEBUG::2014-10-15 06:39:53,142::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-21::DEBUG::2014-10-15 06:39:53,143::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:53,143::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:53,143::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:53,143::__init__::232::IOProcess::(_processLogs) (2501) Got request for method 'glob' >Thread-21::DEBUG::2014-10-15 06:39:53,143::__init__::232::IOProcess::(_processLogs) (2501) Queuing response >Thread-24::DEBUG::2014-10-15 06:39:53,150::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-21::DEBUG::2014-10-15 06:39:53,150::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:53,150::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:53,151::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:53,151::__init__::232::IOProcess::(_processLogs) (2502) Got request for method 'touch' >Thread-21::DEBUG::2014-10-15 06:39:53,191::__init__::232::IOProcess::(_processLogs) (2502) Queuing response >Thread-24::DEBUG::2014-10-15 06:39:53,191::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-21::DEBUG::2014-10-15 06:39:53,192::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:53,192::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:53,192::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:53,192::__init__::232::IOProcess::(_processLogs) (2503) Got request for method 'readfile' >Thread-21::DEBUG::2014-10-15 06:39:53,192::__init__::232::IOProcess::(_processLogs) (2503) Queuing response >Thread-24::DEBUG::2014-10-15 06:39:53,193::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=domain1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-0000000003e1', 'REMOTE_PATH=10.66.128.104:/home/NFS1', 'ROLE=Master', 'SDUUID=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b90f33781d11f6e6b0b3945de3167c0719a458c1'] >Thread-21::DEBUG::2014-10-15 06:39:53,193::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:53,193::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:53,193::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:53,193::__init__::232::IOProcess::(_processLogs) (2504) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:53,194::__init__::232::IOProcess::(_processLogs) (2504) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:53,194::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:53,194::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:53,194::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:53,194::__init__::232::IOProcess::(_processLogs) (2505) Got request for method 'glob' >Thread-21::DEBUG::2014-10-15 06:39:53,194::__init__::232::IOProcess::(_processLogs) (2505) Queuing response >Thread-24::DEBUG::2014-10-15 06:39:53,195::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-24::INFO::2014-10-15 06:39:53,195::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_imageNS already registered >Thread-24::INFO::2014-10-15 06:39:53,195::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_volumeNS already registered >Thread-21::DEBUG::2014-10-15 06:39:53,195::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:53,195::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:53,195::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:53,195::__init__::232::IOProcess::(_processLogs) (2506) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:39:53,196::__init__::232::IOProcess::(_processLogs) (2506) Queuing response >Thread-24::DEBUG::2014-10-15 06:39:53,196::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:39:53,201::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000406047 s, 1.1 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:39:53,202::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:53,202::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:53,202::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:53,202::__init__::232::IOProcess::(_processLogs) (2507) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:39:53,202::__init__::232::IOProcess::(_processLogs) (2507) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:53,203::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:53,203::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:53,203::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:53,203::__init__::232::IOProcess::(_processLogs) (2508) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:53,203::__init__::232::IOProcess::(_processLogs) (2508) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:53,204::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:53,204::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:53,204::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:53,204::__init__::232::IOProcess::(_processLogs) (2509) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:53,204::__init__::232::IOProcess::(_processLogs) (2509) Queuing response >Thread-21::DEBUG::2014-10-15 06:39:53,204::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:39:53,204::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:39:53,205::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:39:53,205::__init__::232::IOProcess::(_processLogs) (2510) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:39:53,205::__init__::232::IOProcess::(_processLogs) (2510) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:53,623::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:53,624::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3402::DEBUG::2014-10-15 06:39:53,625::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:53,626::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:53,626::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3403::DEBUG::2014-10-15 06:39:53,627::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3403::DEBUG::2014-10-15 06:39:53,628::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:39:53,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:54,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114357 s, 89.5 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:55,223::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:55,223::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:39:55,224::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:39:56,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:56,038::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118801 s, 86.2 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:56,650::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:56,651::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3404::DEBUG::2014-10-15 06:39:56,652::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7768ad7-4a88-4fa4-a184-caa57ce98142`::moving from state init -> state preparing >Thread-3404::INFO::2014-10-15 06:39:56,652::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-3404::INFO::2014-10-15 06:39:56,652::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000406047', 'lastCheck': '3.4', 'valid': True}} >Thread-3404::DEBUG::2014-10-15 06:39:56,653::task::1191::Storage.TaskManager.Task::(prepare) Task=`b7768ad7-4a88-4fa4-a184-caa57ce98142`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000406047', 'lastCheck': '3.4', 'valid': True}} >Thread-3404::DEBUG::2014-10-15 06:39:56,653::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7768ad7-4a88-4fa4-a184-caa57ce98142`::moving from state preparing -> state finished >Thread-3404::DEBUG::2014-10-15 06:39:56,653::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3404::DEBUG::2014-10-15 06:39:56,653::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3404::DEBUG::2014-10-15 06:39:56,653::task::993::Storage.TaskManager.Task::(_decref) Task=`b7768ad7-4a88-4fa4-a184-caa57ce98142`::ref 0 aborting False >Thread-3404::ERROR::2014-10-15 06:39:56,654::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-3404::DEBUG::2014-10-15 06:39:56,655::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:56,663::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:56,663::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3405::DEBUG::2014-10-15 06:39:56,664::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3405::DEBUG::2014-10-15 06:39:56,665::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:57,695::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:57,696::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3406::DEBUG::2014-10-15 06:39:57,696::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3406::DEBUG::2014-10-15 06:39:57,697::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d249210-8319-4a01-92c0-55f186c480d5`::moving from state init -> state preparing >Thread-3406::INFO::2014-10-15 06:39:57,697::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3406::INFO::2014-10-15 06:39:57,710::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3406::DEBUG::2014-10-15 06:39:57,710::task::1191::Storage.TaskManager.Task::(prepare) Task=`9d249210-8319-4a01-92c0-55f186c480d5`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3406::DEBUG::2014-10-15 06:39:57,710::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d249210-8319-4a01-92c0-55f186c480d5`::moving from state preparing -> state finished >Thread-3406::DEBUG::2014-10-15 06:39:57,710::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3406::DEBUG::2014-10-15 06:39:57,710::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3406::DEBUG::2014-10-15 06:39:57,710::task::993::Storage.TaskManager.Task::(_decref) Task=`9d249210-8319-4a01-92c0-55f186c480d5`::ref 0 aborting False >Thread-3406::DEBUG::2014-10-15 06:39:57,710::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3406::DEBUG::2014-10-15 06:39:57,711::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:57,718::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:57,719::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3407::DEBUG::2014-10-15 06:39:57,719::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3407::DEBUG::2014-10-15 06:39:57,720::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee5273f7-3772-4be3-93a5-f6d5b610c416`::moving from state init -> state preparing >Thread-3407::INFO::2014-10-15 06:39:57,720::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3407::DEBUG::2014-10-15 06:39:57,720::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`4dec9cbc-99f5-4b04-912f-13f9db83a60d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3407::DEBUG::2014-10-15 06:39:57,720::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3407::DEBUG::2014-10-15 06:39:57,720::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3407::DEBUG::2014-10-15 06:39:57,720::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`4dec9cbc-99f5-4b04-912f-13f9db83a60d`::Granted request >Thread-3407::DEBUG::2014-10-15 06:39:57,721::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ee5273f7-3772-4be3-93a5-f6d5b610c416`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3407::DEBUG::2014-10-15 06:39:57,721::task::993::Storage.TaskManager.Task::(_decref) Task=`ee5273f7-3772-4be3-93a5-f6d5b610c416`::ref 1 aborting False >Thread-3407::INFO::2014-10-15 06:39:57,739::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3407::DEBUG::2014-10-15 06:39:57,739::task::1191::Storage.TaskManager.Task::(prepare) Task=`ee5273f7-3772-4be3-93a5-f6d5b610c416`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3407::DEBUG::2014-10-15 06:39:57,739::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee5273f7-3772-4be3-93a5-f6d5b610c416`::moving from state preparing -> state finished >Thread-3407::DEBUG::2014-10-15 06:39:57,739::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3407::DEBUG::2014-10-15 06:39:57,739::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3407::DEBUG::2014-10-15 06:39:57,739::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3407::DEBUG::2014-10-15 06:39:57,739::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3407::DEBUG::2014-10-15 06:39:57,739::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3407::DEBUG::2014-10-15 06:39:57,740::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3407::DEBUG::2014-10-15 06:39:57,740::task::993::Storage.TaskManager.Task::(_decref) Task=`ee5273f7-3772-4be3-93a5-f6d5b610c416`::ref 0 aborting False >Thread-3407::DEBUG::2014-10-15 06:39:57,740::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3407::DEBUG::2014-10-15 06:39:57,740::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:39:58,047::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:39:58,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0106311 s, 96.3 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:59,755::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:59,756::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3408::DEBUG::2014-10-15 06:39:59,757::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:39:59,758::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:39:59,759::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3409::DEBUG::2014-10-15 06:39:59,760::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3409::DEBUG::2014-10-15 06:39:59,760::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:00,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:00,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174461 s, 58.7 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:00,225::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:00,225::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:00,226::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:40:02,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:02,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017207 s, 59.5 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:02,799::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:02,799::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3410::DEBUG::2014-10-15 06:40:02,800::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:02,801::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:02,802::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3411::DEBUG::2014-10-15 06:40:02,803::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3411::DEBUG::2014-10-15 06:40:02,803::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-21::DEBUG::2014-10-15 06:40:03,206::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:03,206::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:03,206::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:03,206::__init__::232::IOProcess::(_processLogs) (2511) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:40:03,206::__init__::232::IOProcess::(_processLogs) (2511) Queuing response >Thread-24::DEBUG::2014-10-15 06:40:03,206::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:40:03,212::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000388762 s, 1.2 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:40:03,212::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:03,212::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:03,212::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:03,212::__init__::232::IOProcess::(_processLogs) (2512) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:40:03,213::__init__::232::IOProcess::(_processLogs) (2512) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:03,213::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:03,213::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:03,213::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:03,213::__init__::232::IOProcess::(_processLogs) (2513) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:03,213::__init__::232::IOProcess::(_processLogs) (2513) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:03,214::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:03,214::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:03,214::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:03,214::__init__::232::IOProcess::(_processLogs) (2514) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:03,214::__init__::232::IOProcess::(_processLogs) (2514) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:03,215::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:03,215::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:03,215::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:03,215::__init__::232::IOProcess::(_processLogs) (2515) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:03,215::__init__::232::IOProcess::(_processLogs) (2515) Queuing response >Dummy-838::DEBUG::2014-10-15 06:40:04,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:04,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0162948 s, 62.8 MB/s\n'; <rc> = 0 >Thread-167::DEBUG::2014-10-15 06:40:05,171::task::595::Storage.TaskManager.Task::(_updateState) Task=`a1d43d66-3c98-497f-a91e-56b0dc018681`::moving from state init -> state preparing >Thread-167::INFO::2014-10-15 06:40:05,171::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-21::DEBUG::2014-10-15 06:40:05,172::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:05,172::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:05,172::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:05,172::__init__::232::IOProcess::(_processLogs) (2516) Got request for method 'stat' >Thread-21::DEBUG::2014-10-15 06:40:05,173::__init__::232::IOProcess::(_processLogs) (2516) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:05,173::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:05,173::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:05,174::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:05,174::__init__::232::IOProcess::(_processLogs) (2517) Got request for method 'stat' >Thread-21::DEBUG::2014-10-15 06:40:05,174::__init__::232::IOProcess::(_processLogs) (2517) Queuing response >Thread-167::INFO::2014-10-15 06:40:05,174::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-167::DEBUG::2014-10-15 06:40:05,174::task::1191::Storage.TaskManager.Task::(prepare) Task=`a1d43d66-3c98-497f-a91e-56b0dc018681`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-167::DEBUG::2014-10-15 06:40:05,174::task::595::Storage.TaskManager.Task::(_updateState) Task=`a1d43d66-3c98-497f-a91e-56b0dc018681`::moving from state preparing -> state finished >Thread-167::DEBUG::2014-10-15 06:40:05,174::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-167::DEBUG::2014-10-15 06:40:05,174::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-167::DEBUG::2014-10-15 06:40:05,174::task::993::Storage.TaskManager.Task::(_decref) Task=`a1d43d66-3c98-497f-a91e-56b0dc018681`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:05,232::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:05,232::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:05,232::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:05,824::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:05,825::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3412::DEBUG::2014-10-15 06:40:05,826::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:05,827::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:05,827::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3413::DEBUG::2014-10-15 06:40:05,828::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3413::DEBUG::2014-10-15 06:40:05,828::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:06,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:06,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115586 s, 88.6 MB/s\n'; <rc> = 0 >VM Channels Listener::DEBUG::2014-10-15 06:40:07,418::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 47. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:07,761::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:07,762::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3414::DEBUG::2014-10-15 06:40:07,762::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3414::DEBUG::2014-10-15 06:40:07,763::task::595::Storage.TaskManager.Task::(_updateState) Task=`271773c5-0653-46ad-9156-ca03272dc989`::moving from state init -> state preparing >Thread-3414::INFO::2014-10-15 06:40:07,763::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3414::INFO::2014-10-15 06:40:07,776::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3414::DEBUG::2014-10-15 06:40:07,776::task::1191::Storage.TaskManager.Task::(prepare) Task=`271773c5-0653-46ad-9156-ca03272dc989`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3414::DEBUG::2014-10-15 06:40:07,776::task::595::Storage.TaskManager.Task::(_updateState) Task=`271773c5-0653-46ad-9156-ca03272dc989`::moving from state preparing -> state finished >Thread-3414::DEBUG::2014-10-15 06:40:07,776::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3414::DEBUG::2014-10-15 06:40:07,776::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3414::DEBUG::2014-10-15 06:40:07,776::task::993::Storage.TaskManager.Task::(_decref) Task=`271773c5-0653-46ad-9156-ca03272dc989`::ref 0 aborting False >Thread-3414::DEBUG::2014-10-15 06:40:07,776::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3414::DEBUG::2014-10-15 06:40:07,776::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:07,784::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:07,785::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3415::DEBUG::2014-10-15 06:40:07,785::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3415::DEBUG::2014-10-15 06:40:07,786::task::595::Storage.TaskManager.Task::(_updateState) Task=`d19850b1-026c-4423-9490-7ca7db242638`::moving from state init -> state preparing >Thread-3415::INFO::2014-10-15 06:40:07,786::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3415::DEBUG::2014-10-15 06:40:07,786::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`cdf7c7fb-07f0-45a5-a34d-0e47d0d8287f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3415::DEBUG::2014-10-15 06:40:07,786::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3415::DEBUG::2014-10-15 06:40:07,786::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3415::DEBUG::2014-10-15 06:40:07,787::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`cdf7c7fb-07f0-45a5-a34d-0e47d0d8287f`::Granted request >Thread-3415::DEBUG::2014-10-15 06:40:07,787::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d19850b1-026c-4423-9490-7ca7db242638`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3415::DEBUG::2014-10-15 06:40:07,787::task::993::Storage.TaskManager.Task::(_decref) Task=`d19850b1-026c-4423-9490-7ca7db242638`::ref 1 aborting False >Thread-3415::INFO::2014-10-15 06:40:07,805::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3415::DEBUG::2014-10-15 06:40:07,805::task::1191::Storage.TaskManager.Task::(prepare) Task=`d19850b1-026c-4423-9490-7ca7db242638`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3415::DEBUG::2014-10-15 06:40:07,805::task::595::Storage.TaskManager.Task::(_updateState) Task=`d19850b1-026c-4423-9490-7ca7db242638`::moving from state preparing -> state finished >Thread-3415::DEBUG::2014-10-15 06:40:07,805::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3415::DEBUG::2014-10-15 06:40:07,806::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3415::DEBUG::2014-10-15 06:40:07,806::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3415::DEBUG::2014-10-15 06:40:07,806::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3415::DEBUG::2014-10-15 06:40:07,806::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3415::DEBUG::2014-10-15 06:40:07,806::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3415::DEBUG::2014-10-15 06:40:07,806::task::993::Storage.TaskManager.Task::(_decref) Task=`d19850b1-026c-4423-9490-7ca7db242638`::ref 0 aborting False >Thread-3415::DEBUG::2014-10-15 06:40:07,806::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3415::DEBUG::2014-10-15 06:40:07,807::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:08,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:08,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0107072 s, 95.6 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:08,852::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:08,852::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3416::DEBUG::2014-10-15 06:40:08,853::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:08,855::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:08,855::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3417::DEBUG::2014-10-15 06:40:08,856::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3417::DEBUG::2014-10-15 06:40:08,856::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:10,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:10,236::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116098 s, 88.2 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:10,239::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:10,241::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:10,242::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:11,879::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:11,880::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3418::DEBUG::2014-10-15 06:40:11,881::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a33af7b-8436-4f95-83f1-76fb050a0c36`::moving from state init -> state preparing >Thread-3418::INFO::2014-10-15 06:40:11,881::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-3418::INFO::2014-10-15 06:40:11,882::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000388762', 'lastCheck': '8.7', 'valid': True}} >Thread-3418::DEBUG::2014-10-15 06:40:11,882::task::1191::Storage.TaskManager.Task::(prepare) Task=`8a33af7b-8436-4f95-83f1-76fb050a0c36`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000388762', 'lastCheck': '8.7', 'valid': True}} >Thread-3418::DEBUG::2014-10-15 06:40:11,882::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a33af7b-8436-4f95-83f1-76fb050a0c36`::moving from state preparing -> state finished >Thread-3418::DEBUG::2014-10-15 06:40:11,882::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3418::DEBUG::2014-10-15 06:40:11,882::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3418::DEBUG::2014-10-15 06:40:11,882::task::993::Storage.TaskManager.Task::(_decref) Task=`8a33af7b-8436-4f95-83f1-76fb050a0c36`::ref 0 aborting False >Thread-3418::ERROR::2014-10-15 06:40:11,883::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-3418::DEBUG::2014-10-15 06:40:11,883::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:11,891::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:11,891::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3419::DEBUG::2014-10-15 06:40:11,892::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3419::DEBUG::2014-10-15 06:40:11,893::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:12,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:12,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113243 s, 90.4 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:40:13,216::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:13,216::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:13,217::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:13,217::__init__::232::IOProcess::(_processLogs) (2518) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:40:13,217::__init__::232::IOProcess::(_processLogs) (2518) Queuing response >Thread-24::DEBUG::2014-10-15 06:40:13,217::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:40:13,223::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000367624 s, 1.2 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:40:13,224::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:13,224::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:13,224::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:13,224::__init__::232::IOProcess::(_processLogs) (2519) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:40:13,224::__init__::232::IOProcess::(_processLogs) (2519) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:13,225::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:13,225::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:13,225::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:13,225::__init__::232::IOProcess::(_processLogs) (2520) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:13,225::__init__::232::IOProcess::(_processLogs) (2520) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:13,225::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:13,225::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:13,225::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:13,226::__init__::232::IOProcess::(_processLogs) (2521) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:13,226::__init__::232::IOProcess::(_processLogs) (2521) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:13,226::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:13,226::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:13,226::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:13,226::__init__::232::IOProcess::(_processLogs) (2522) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:13,226::__init__::232::IOProcess::(_processLogs) (2522) Queuing response >Dummy-838::DEBUG::2014-10-15 06:40:14,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:14,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169642 s, 60.4 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:14,983::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:14,984::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3420::DEBUG::2014-10-15 06:40:14,985::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:14,986::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:14,987::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3421::DEBUG::2014-10-15 06:40:14,988::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3421::DEBUG::2014-10-15 06:40:14,988::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:15,248::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:15,249::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:15,249::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:40:16,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:16,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0106707 s, 96.0 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:17,825::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:17,826::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3422::DEBUG::2014-10-15 06:40:17,826::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3422::DEBUG::2014-10-15 06:40:17,827::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f495cb2-4ce5-499b-94a9-6dd7d05abb34`::moving from state init -> state preparing >Thread-3422::INFO::2014-10-15 06:40:17,827::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3422::INFO::2014-10-15 06:40:17,839::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3422::DEBUG::2014-10-15 06:40:17,839::task::1191::Storage.TaskManager.Task::(prepare) Task=`0f495cb2-4ce5-499b-94a9-6dd7d05abb34`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3422::DEBUG::2014-10-15 06:40:17,839::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f495cb2-4ce5-499b-94a9-6dd7d05abb34`::moving from state preparing -> state finished >Thread-3422::DEBUG::2014-10-15 06:40:17,840::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3422::DEBUG::2014-10-15 06:40:17,840::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3422::DEBUG::2014-10-15 06:40:17,840::task::993::Storage.TaskManager.Task::(_decref) Task=`0f495cb2-4ce5-499b-94a9-6dd7d05abb34`::ref 0 aborting False >Thread-3422::DEBUG::2014-10-15 06:40:17,840::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3422::DEBUG::2014-10-15 06:40:17,840::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:17,848::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-3423::DEBUG::2014-10-15 06:40:17,848::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >JsonRpcServer::DEBUG::2014-10-15 06:40:17,848::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3423::DEBUG::2014-10-15 06:40:17,849::task::595::Storage.TaskManager.Task::(_updateState) Task=`274da6e2-5f98-4480-825d-88551fe10875`::moving from state init -> state preparing >Thread-3423::INFO::2014-10-15 06:40:17,849::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3423::DEBUG::2014-10-15 06:40:17,850::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`f5a90360-d169-4002-a81e-25eda02a35d4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3423::DEBUG::2014-10-15 06:40:17,850::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3423::DEBUG::2014-10-15 06:40:17,850::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3423::DEBUG::2014-10-15 06:40:17,850::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`f5a90360-d169-4002-a81e-25eda02a35d4`::Granted request >Thread-3423::DEBUG::2014-10-15 06:40:17,850::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`274da6e2-5f98-4480-825d-88551fe10875`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3423::DEBUG::2014-10-15 06:40:17,850::task::993::Storage.TaskManager.Task::(_decref) Task=`274da6e2-5f98-4480-825d-88551fe10875`::ref 1 aborting False >Thread-3423::INFO::2014-10-15 06:40:17,868::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3423::DEBUG::2014-10-15 06:40:17,868::task::1191::Storage.TaskManager.Task::(prepare) Task=`274da6e2-5f98-4480-825d-88551fe10875`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3423::DEBUG::2014-10-15 06:40:17,868::task::595::Storage.TaskManager.Task::(_updateState) Task=`274da6e2-5f98-4480-825d-88551fe10875`::moving from state preparing -> state finished >Thread-3423::DEBUG::2014-10-15 06:40:17,868::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3423::DEBUG::2014-10-15 06:40:17,868::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3423::DEBUG::2014-10-15 06:40:17,869::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3423::DEBUG::2014-10-15 06:40:17,869::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3423::DEBUG::2014-10-15 06:40:17,869::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3423::DEBUG::2014-10-15 06:40:17,869::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3423::DEBUG::2014-10-15 06:40:17,869::task::993::Storage.TaskManager.Task::(_decref) Task=`274da6e2-5f98-4480-825d-88551fe10875`::ref 0 aborting False >Thread-3423::DEBUG::2014-10-15 06:40:17,869::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3423::DEBUG::2014-10-15 06:40:17,869::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:18,026::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:18,026::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3424::DEBUG::2014-10-15 06:40:18,027::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:18,028::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:18,029::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3425::DEBUG::2014-10-15 06:40:18,030::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3425::DEBUG::2014-10-15 06:40:18,030::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:18,402::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:18,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0105816 s, 96.8 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:20,255::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:20,255::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:20,256::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:40:20,426::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:20,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113999 s, 89.8 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:21,048::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:21,049::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3426::DEBUG::2014-10-15 06:40:21,050::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:21,051::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:21,051::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3427::DEBUG::2014-10-15 06:40:21,052::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3427::DEBUG::2014-10-15 06:40:21,053::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:22,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:22,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113522 s, 90.2 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:40:23,227::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:23,228::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:23,228::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:23,228::__init__::232::IOProcess::(_processLogs) (2523) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:40:23,228::__init__::232::IOProcess::(_processLogs) (2523) Queuing response >Thread-24::DEBUG::2014-10-15 06:40:23,228::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:40:23,234::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000465408 s, 986 kB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:40:23,235::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:23,235::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:23,235::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:23,235::__init__::232::IOProcess::(_processLogs) (2524) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:40:23,235::__init__::232::IOProcess::(_processLogs) (2524) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:23,236::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:23,236::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:23,236::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:23,236::__init__::232::IOProcess::(_processLogs) (2525) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:23,236::__init__::232::IOProcess::(_processLogs) (2525) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:23,237::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:23,237::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:23,237::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:23,237::__init__::232::IOProcess::(_processLogs) (2526) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:23,237::__init__::232::IOProcess::(_processLogs) (2526) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:23,238::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:23,238::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:23,238::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:23,238::__init__::232::IOProcess::(_processLogs) (2527) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:23,238::__init__::232::IOProcess::(_processLogs) (2527) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:24,074::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:24,075::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3428::DEBUG::2014-10-15 06:40:24,076::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:24,077::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:24,078::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3429::DEBUG::2014-10-15 06:40:24,079::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3429::DEBUG::2014-10-15 06:40:24,079::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:24,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:24,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115433 s, 88.7 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:25,261::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:25,262::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:25,262::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:40:26,503::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:26,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114215 s, 89.7 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:27,099::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:27,100::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3430::DEBUG::2014-10-15 06:40:27,101::task::595::Storage.TaskManager.Task::(_updateState) Task=`b18dd7d9-47bb-46d4-8bda-96a18fe9a043`::moving from state init -> state preparing >Thread-3430::INFO::2014-10-15 06:40:27,101::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-3430::INFO::2014-10-15 06:40:27,101::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000465408', 'lastCheck': '3.9', 'valid': True}} >Thread-3430::DEBUG::2014-10-15 06:40:27,102::task::1191::Storage.TaskManager.Task::(prepare) Task=`b18dd7d9-47bb-46d4-8bda-96a18fe9a043`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000465408', 'lastCheck': '3.9', 'valid': True}} >Thread-3430::DEBUG::2014-10-15 06:40:27,102::task::595::Storage.TaskManager.Task::(_updateState) Task=`b18dd7d9-47bb-46d4-8bda-96a18fe9a043`::moving from state preparing -> state finished >Thread-3430::DEBUG::2014-10-15 06:40:27,102::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3430::DEBUG::2014-10-15 06:40:27,102::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3430::DEBUG::2014-10-15 06:40:27,102::task::993::Storage.TaskManager.Task::(_decref) Task=`b18dd7d9-47bb-46d4-8bda-96a18fe9a043`::ref 0 aborting False >Thread-3430::ERROR::2014-10-15 06:40:27,103::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-3430::DEBUG::2014-10-15 06:40:27,103::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:27,111::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:27,111::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3431::DEBUG::2014-10-15 06:40:27,112::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3431::DEBUG::2014-10-15 06:40:27,113::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:27,899::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-3432::DEBUG::2014-10-15 06:40:27,899::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >JsonRpcServer::DEBUG::2014-10-15 06:40:27,899::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3432::DEBUG::2014-10-15 06:40:27,901::task::595::Storage.TaskManager.Task::(_updateState) Task=`f215161d-66c3-4a82-8300-d46b12269b2a`::moving from state init -> state preparing >Thread-3432::INFO::2014-10-15 06:40:27,901::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3432::INFO::2014-10-15 06:40:27,914::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3432::DEBUG::2014-10-15 06:40:27,914::task::1191::Storage.TaskManager.Task::(prepare) Task=`f215161d-66c3-4a82-8300-d46b12269b2a`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3432::DEBUG::2014-10-15 06:40:27,914::task::595::Storage.TaskManager.Task::(_updateState) Task=`f215161d-66c3-4a82-8300-d46b12269b2a`::moving from state preparing -> state finished >Thread-3432::DEBUG::2014-10-15 06:40:27,914::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3432::DEBUG::2014-10-15 06:40:27,914::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3432::DEBUG::2014-10-15 06:40:27,914::task::993::Storage.TaskManager.Task::(_decref) Task=`f215161d-66c3-4a82-8300-d46b12269b2a`::ref 0 aborting False >Thread-3432::DEBUG::2014-10-15 06:40:27,914::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3432::DEBUG::2014-10-15 06:40:27,915::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:27,922::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:27,923::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3433::DEBUG::2014-10-15 06:40:27,923::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3433::DEBUG::2014-10-15 06:40:27,923::task::595::Storage.TaskManager.Task::(_updateState) Task=`08552ae4-21dc-4b06-99e9-8276c1d6b1ad`::moving from state init -> state preparing >Thread-3433::INFO::2014-10-15 06:40:27,924::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3433::DEBUG::2014-10-15 06:40:27,924::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`9f9c2ae5-2788-469e-92b7-f72105748dac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3433::DEBUG::2014-10-15 06:40:27,924::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3433::DEBUG::2014-10-15 06:40:27,924::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3433::DEBUG::2014-10-15 06:40:27,924::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`9f9c2ae5-2788-469e-92b7-f72105748dac`::Granted request >Thread-3433::DEBUG::2014-10-15 06:40:27,925::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`08552ae4-21dc-4b06-99e9-8276c1d6b1ad`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3433::DEBUG::2014-10-15 06:40:27,925::task::993::Storage.TaskManager.Task::(_decref) Task=`08552ae4-21dc-4b06-99e9-8276c1d6b1ad`::ref 1 aborting False >Thread-3433::INFO::2014-10-15 06:40:27,942::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3433::DEBUG::2014-10-15 06:40:27,943::task::1191::Storage.TaskManager.Task::(prepare) Task=`08552ae4-21dc-4b06-99e9-8276c1d6b1ad`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3433::DEBUG::2014-10-15 06:40:27,943::task::595::Storage.TaskManager.Task::(_updateState) Task=`08552ae4-21dc-4b06-99e9-8276c1d6b1ad`::moving from state preparing -> state finished >Thread-3433::DEBUG::2014-10-15 06:40:27,943::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3433::DEBUG::2014-10-15 06:40:27,943::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3433::DEBUG::2014-10-15 06:40:27,943::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3433::DEBUG::2014-10-15 06:40:27,943::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3433::DEBUG::2014-10-15 06:40:27,943::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3433::DEBUG::2014-10-15 06:40:27,943::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3433::DEBUG::2014-10-15 06:40:27,943::task::993::Storage.TaskManager.Task::(_decref) Task=`08552ae4-21dc-4b06-99e9-8276c1d6b1ad`::ref 0 aborting False >Thread-3433::DEBUG::2014-10-15 06:40:27,944::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3433::DEBUG::2014-10-15 06:40:27,944::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:28,529::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:28,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0107676 s, 95.1 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:30,209::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:30,209::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3434::DEBUG::2014-10-15 06:40:30,210::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:30,211::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:30,212::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3435::DEBUG::2014-10-15 06:40:30,213::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3435::DEBUG::2014-10-15 06:40:30,213::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:30,265::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:30,266::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:30,266::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:40:30,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:30,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0113981 s, 89.8 MB/s\n'; <rc> = 0 >Dummy-838::DEBUG::2014-10-15 06:40:32,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:32,596::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114964 s, 89.1 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:40:33,239::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:33,239::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:33,240::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-21::DEBUG::2014-10-15 06:40:33,240::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:33,241::__init__::232::IOProcess::(_processLogs) (2528) Got request for method 'statvfs' >JsonRpcServer::DEBUG::2014-10-15 06:40:33,241::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-21::DEBUG::2014-10-15 06:40:33,242::__init__::232::IOProcess::(_processLogs) (2528) Queuing response >Thread-24::DEBUG::2014-10-15 06:40:33,244::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-3436::DEBUG::2014-10-15 06:40:33,249::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-24::DEBUG::2014-10-15 06:40:33,250::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00036286 s, 1.3 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:40:33,251::__init__::232::IOProcess::(_processLogs) Receiving request... >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:33,251::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-21::DEBUG::2014-10-15 06:40:33,251::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:33,252::__init__::232::IOProcess::(_processLogs) Extracting request information... >JsonRpcServer::DEBUG::2014-10-15 06:40:33,253::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-21::DEBUG::2014-10-15 06:40:33,253::__init__::232::IOProcess::(_processLogs) (2529) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:40:33,254::__init__::232::IOProcess::(_processLogs) (2529) Queuing response >Thread-3437::DEBUG::2014-10-15 06:40:33,255::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-21::DEBUG::2014-10-15 06:40:33,256::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:33,257::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-3437::DEBUG::2014-10-15 06:40:33,257::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-21::DEBUG::2014-10-15 06:40:33,257::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:33,258::__init__::232::IOProcess::(_processLogs) (2530) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:33,258::__init__::232::IOProcess::(_processLogs) (2530) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:33,259::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:33,259::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:33,259::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:33,259::__init__::232::IOProcess::(_processLogs) (2531) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:33,259::__init__::232::IOProcess::(_processLogs) (2531) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:33,259::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:33,259::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:33,260::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:33,260::__init__::232::IOProcess::(_processLogs) (2532) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:33,260::__init__::232::IOProcess::(_processLogs) (2532) Queuing response >Dummy-838::DEBUG::2014-10-15 06:40:34,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:34,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114312 s, 89.6 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:35,267::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:35,268::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:35,268::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:36,285::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:36,285::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3438::DEBUG::2014-10-15 06:40:36,286::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:36,287::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:36,288::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3439::DEBUG::2014-10-15 06:40:36,289::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3439::DEBUG::2014-10-15 06:40:36,289::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:36,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:36,647::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114127 s, 89.7 MB/s\n'; <rc> = 0 >VM Channels Listener::DEBUG::2014-10-15 06:40:37,449::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 47. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:37,958::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-3440::DEBUG::2014-10-15 06:40:37,959::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >JsonRpcServer::DEBUG::2014-10-15 06:40:37,959::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3440::DEBUG::2014-10-15 06:40:37,960::task::595::Storage.TaskManager.Task::(_updateState) Task=`62e51a70-d01c-40aa-b0a8-c65181efe1a8`::moving from state init -> state preparing >Thread-3440::INFO::2014-10-15 06:40:37,960::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3440::INFO::2014-10-15 06:40:37,973::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3440::DEBUG::2014-10-15 06:40:37,973::task::1191::Storage.TaskManager.Task::(prepare) Task=`62e51a70-d01c-40aa-b0a8-c65181efe1a8`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3440::DEBUG::2014-10-15 06:40:37,973::task::595::Storage.TaskManager.Task::(_updateState) Task=`62e51a70-d01c-40aa-b0a8-c65181efe1a8`::moving from state preparing -> state finished >Thread-3440::DEBUG::2014-10-15 06:40:37,973::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3440::DEBUG::2014-10-15 06:40:37,973::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3440::DEBUG::2014-10-15 06:40:37,973::task::993::Storage.TaskManager.Task::(_decref) Task=`62e51a70-d01c-40aa-b0a8-c65181efe1a8`::ref 0 aborting False >Thread-3440::DEBUG::2014-10-15 06:40:37,974::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3440::DEBUG::2014-10-15 06:40:37,974::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:37,982::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:37,982::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3441::DEBUG::2014-10-15 06:40:37,982::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3441::DEBUG::2014-10-15 06:40:37,983::task::595::Storage.TaskManager.Task::(_updateState) Task=`055523b1-745f-4b14-b6fe-78b1c86f89bb`::moving from state init -> state preparing >Thread-3441::INFO::2014-10-15 06:40:37,983::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3441::DEBUG::2014-10-15 06:40:37,983::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`5b3ea70a-a2e7-4fd7-afa6-a82f396f775b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3441::DEBUG::2014-10-15 06:40:37,983::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3441::DEBUG::2014-10-15 06:40:37,984::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3441::DEBUG::2014-10-15 06:40:37,984::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`5b3ea70a-a2e7-4fd7-afa6-a82f396f775b`::Granted request >Thread-3441::DEBUG::2014-10-15 06:40:37,984::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`055523b1-745f-4b14-b6fe-78b1c86f89bb`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3441::DEBUG::2014-10-15 06:40:37,984::task::993::Storage.TaskManager.Task::(_decref) Task=`055523b1-745f-4b14-b6fe-78b1c86f89bb`::ref 1 aborting False >Thread-3441::INFO::2014-10-15 06:40:38,002::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3441::DEBUG::2014-10-15 06:40:38,002::task::1191::Storage.TaskManager.Task::(prepare) Task=`055523b1-745f-4b14-b6fe-78b1c86f89bb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3441::DEBUG::2014-10-15 06:40:38,002::task::595::Storage.TaskManager.Task::(_updateState) Task=`055523b1-745f-4b14-b6fe-78b1c86f89bb`::moving from state preparing -> state finished >Thread-3441::DEBUG::2014-10-15 06:40:38,002::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3441::DEBUG::2014-10-15 06:40:38,003::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3441::DEBUG::2014-10-15 06:40:38,003::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3441::DEBUG::2014-10-15 06:40:38,003::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3441::DEBUG::2014-10-15 06:40:38,003::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3441::DEBUG::2014-10-15 06:40:38,003::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3441::DEBUG::2014-10-15 06:40:38,003::task::993::Storage.TaskManager.Task::(_decref) Task=`055523b1-745f-4b14-b6fe-78b1c86f89bb`::ref 0 aborting False >Thread-3441::DEBUG::2014-10-15 06:40:38,003::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3441::DEBUG::2014-10-15 06:40:38,004::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:38,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:38,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0105902 s, 96.7 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:39,314::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:39,315::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3442::DEBUG::2014-10-15 06:40:39,316::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:39,317::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:39,318::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3443::DEBUG::2014-10-15 06:40:39,319::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3443::DEBUG::2014-10-15 06:40:39,319::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:40,274::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:40,274::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:40,274::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:40:40,681::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:40,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117521 s, 87.1 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:42,351::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:42,351::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3444::DEBUG::2014-10-15 06:40:42,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b617000-ad6f-4e7a-9ced-f32cd351a796`::moving from state init -> state preparing >Thread-3444::INFO::2014-10-15 06:40:42,352::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-3444::INFO::2014-10-15 06:40:42,353::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00036286', 'lastCheck': '9.1', 'valid': True}} >Thread-3444::DEBUG::2014-10-15 06:40:42,353::task::1191::Storage.TaskManager.Task::(prepare) Task=`2b617000-ad6f-4e7a-9ced-f32cd351a796`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00036286', 'lastCheck': '9.1', 'valid': True}} >Thread-3444::DEBUG::2014-10-15 06:40:42,353::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b617000-ad6f-4e7a-9ced-f32cd351a796`::moving from state preparing -> state finished >Thread-3444::DEBUG::2014-10-15 06:40:42,353::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3444::DEBUG::2014-10-15 06:40:42,353::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3444::DEBUG::2014-10-15 06:40:42,353::task::993::Storage.TaskManager.Task::(_decref) Task=`2b617000-ad6f-4e7a-9ced-f32cd351a796`::ref 0 aborting False >Thread-3444::ERROR::2014-10-15 06:40:42,354::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-3444::DEBUG::2014-10-15 06:40:42,355::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:42,363::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:42,364::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3445::DEBUG::2014-10-15 06:40:42,365::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3445::DEBUG::2014-10-15 06:40:42,365::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:42,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:42,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115888 s, 88.4 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:40:43,261::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:43,261::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:43,261::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:43,261::__init__::232::IOProcess::(_processLogs) (2533) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:40:43,261::__init__::232::IOProcess::(_processLogs) (2533) Queuing response >Thread-24::DEBUG::2014-10-15 06:40:43,262::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:40:43,267::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000392131 s, 1.2 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:40:43,268::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:43,268::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:43,268::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:43,268::__init__::232::IOProcess::(_processLogs) (2534) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:40:43,269::__init__::232::IOProcess::(_processLogs) (2534) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:43,269::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:43,269::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:43,269::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:43,269::__init__::232::IOProcess::(_processLogs) (2535) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:43,269::__init__::232::IOProcess::(_processLogs) (2535) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:43,270::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:43,270::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:43,270::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:43,270::__init__::232::IOProcess::(_processLogs) (2536) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:43,270::__init__::232::IOProcess::(_processLogs) (2536) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:43,271::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:43,271::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:43,271::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:43,271::__init__::232::IOProcess::(_processLogs) (2537) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:43,271::__init__::232::IOProcess::(_processLogs) (2537) Queuing response >Dummy-838::DEBUG::2014-10-15 06:40:44,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:44,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178684 s, 57.3 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:45,280::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:45,281::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:45,281::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:45,457::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:45,457::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3446::DEBUG::2014-10-15 06:40:45,458::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:45,459::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:45,460::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3447::DEBUG::2014-10-15 06:40:45,461::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3447::DEBUG::2014-10-15 06:40:45,461::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:46,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:46,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.014928 s, 68.6 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:48,019::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-3448::DEBUG::2014-10-15 06:40:48,020::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >JsonRpcServer::DEBUG::2014-10-15 06:40:48,020::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3448::DEBUG::2014-10-15 06:40:48,021::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd4f2f9b-1471-4d9b-8abd-ff6cd9ddadf0`::moving from state init -> state preparing >Thread-3448::INFO::2014-10-15 06:40:48,021::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3448::INFO::2014-10-15 06:40:48,034::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3448::DEBUG::2014-10-15 06:40:48,034::task::1191::Storage.TaskManager.Task::(prepare) Task=`bd4f2f9b-1471-4d9b-8abd-ff6cd9ddadf0`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3448::DEBUG::2014-10-15 06:40:48,034::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd4f2f9b-1471-4d9b-8abd-ff6cd9ddadf0`::moving from state preparing -> state finished >Thread-3448::DEBUG::2014-10-15 06:40:48,034::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3448::DEBUG::2014-10-15 06:40:48,034::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3448::DEBUG::2014-10-15 06:40:48,034::task::993::Storage.TaskManager.Task::(_decref) Task=`bd4f2f9b-1471-4d9b-8abd-ff6cd9ddadf0`::ref 0 aborting False >Thread-3448::DEBUG::2014-10-15 06:40:48,034::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3448::DEBUG::2014-10-15 06:40:48,035::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:48,041::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:48,041::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3449::DEBUG::2014-10-15 06:40:48,041::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3449::DEBUG::2014-10-15 06:40:48,042::task::595::Storage.TaskManager.Task::(_updateState) Task=`35dc47df-c911-4e11-8e46-a3805b0a2c3d`::moving from state init -> state preparing >Thread-3449::INFO::2014-10-15 06:40:48,042::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3449::DEBUG::2014-10-15 06:40:48,042::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`db616876-d23a-472a-8e17-ddf8edd5dae5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3449::DEBUG::2014-10-15 06:40:48,042::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3449::DEBUG::2014-10-15 06:40:48,043::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3449::DEBUG::2014-10-15 06:40:48,043::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`db616876-d23a-472a-8e17-ddf8edd5dae5`::Granted request >Thread-3449::DEBUG::2014-10-15 06:40:48,043::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`35dc47df-c911-4e11-8e46-a3805b0a2c3d`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3449::DEBUG::2014-10-15 06:40:48,043::task::993::Storage.TaskManager.Task::(_decref) Task=`35dc47df-c911-4e11-8e46-a3805b0a2c3d`::ref 1 aborting False >Thread-3449::INFO::2014-10-15 06:40:48,062::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3449::DEBUG::2014-10-15 06:40:48,062::task::1191::Storage.TaskManager.Task::(prepare) Task=`35dc47df-c911-4e11-8e46-a3805b0a2c3d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3449::DEBUG::2014-10-15 06:40:48,062::task::595::Storage.TaskManager.Task::(_updateState) Task=`35dc47df-c911-4e11-8e46-a3805b0a2c3d`::moving from state preparing -> state finished >Thread-3449::DEBUG::2014-10-15 06:40:48,062::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3449::DEBUG::2014-10-15 06:40:48,062::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3449::DEBUG::2014-10-15 06:40:48,062::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3449::DEBUG::2014-10-15 06:40:48,063::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3449::DEBUG::2014-10-15 06:40:48,063::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3449::DEBUG::2014-10-15 06:40:48,063::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3449::DEBUG::2014-10-15 06:40:48,063::task::993::Storage.TaskManager.Task::(_decref) Task=`35dc47df-c911-4e11-8e46-a3805b0a2c3d`::ref 0 aborting False >Thread-3449::DEBUG::2014-10-15 06:40:48,063::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3449::DEBUG::2014-10-15 06:40:48,063::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:48,498::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:48,498::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3450::DEBUG::2014-10-15 06:40:48,499::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:48,500::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:48,501::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3451::DEBUG::2014-10-15 06:40:48,502::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3451::DEBUG::2014-10-15 06:40:48,502::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:48,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:48,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0413697 s, 24.8 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:50,287::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:50,287::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:50,288::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:40:50,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:50,869::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01341 s, 76.4 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:51,524::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:51,525::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3452::DEBUG::2014-10-15 06:40:51,526::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:51,527::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:51,528::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3453::DEBUG::2014-10-15 06:40:51,528::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3453::DEBUG::2014-10-15 06:40:51,529::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:52,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:52,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0147302 s, 69.5 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:40:53,272::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:53,272::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:53,273::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:53,273::__init__::232::IOProcess::(_processLogs) (2538) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:40:53,273::__init__::232::IOProcess::(_processLogs) (2538) Queuing response >Thread-24::DEBUG::2014-10-15 06:40:53,273::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:40:53,279::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000471393 s, 974 kB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:40:53,280::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:53,280::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:53,280::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:53,280::__init__::232::IOProcess::(_processLogs) (2539) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:40:53,280::__init__::232::IOProcess::(_processLogs) (2539) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:53,281::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:53,281::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:53,281::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:53,281::__init__::232::IOProcess::(_processLogs) (2540) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:53,281::__init__::232::IOProcess::(_processLogs) (2540) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:53,282::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:53,282::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:53,282::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:53,282::__init__::232::IOProcess::(_processLogs) (2541) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:53,282::__init__::232::IOProcess::(_processLogs) (2541) Queuing response >Thread-21::DEBUG::2014-10-15 06:40:53,282::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:40:53,282::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:40:53,282::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:40:53,283::__init__::232::IOProcess::(_processLogs) (2542) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:40:53,283::__init__::232::IOProcess::(_processLogs) (2542) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:54,551::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:54,552::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3454::DEBUG::2014-10-15 06:40:54,553::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:54,554::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:54,554::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3455::DEBUG::2014-10-15 06:40:54,555::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3455::DEBUG::2014-10-15 06:40:54,556::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:54,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:54,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119243 s, 85.9 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:55,294::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:55,294::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:40:55,295::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:40:56,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:56,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0129423 s, 79.1 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:57,575::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:57,576::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3456::DEBUG::2014-10-15 06:40:57,577::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f592705-1eff-4de2-abf1-f9a3f754dfe4`::moving from state init -> state preparing >Thread-3456::INFO::2014-10-15 06:40:57,577::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-3456::INFO::2014-10-15 06:40:57,578::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000471393', 'lastCheck': '4.3', 'valid': True}} >Thread-3456::DEBUG::2014-10-15 06:40:57,578::task::1191::Storage.TaskManager.Task::(prepare) Task=`5f592705-1eff-4de2-abf1-f9a3f754dfe4`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000471393', 'lastCheck': '4.3', 'valid': True}} >Thread-3456::DEBUG::2014-10-15 06:40:57,578::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f592705-1eff-4de2-abf1-f9a3f754dfe4`::moving from state preparing -> state finished >Thread-3456::DEBUG::2014-10-15 06:40:57,578::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3456::DEBUG::2014-10-15 06:40:57,578::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3456::DEBUG::2014-10-15 06:40:57,578::task::993::Storage.TaskManager.Task::(_decref) Task=`5f592705-1eff-4de2-abf1-f9a3f754dfe4`::ref 0 aborting False >Thread-3456::ERROR::2014-10-15 06:40:57,581::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-3456::DEBUG::2014-10-15 06:40:57,584::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:57,598::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:57,599::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3457::DEBUG::2014-10-15 06:40:57,602::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3457::DEBUG::2014-10-15 06:40:57,604::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:58,086::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:58,087::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3458::DEBUG::2014-10-15 06:40:58,087::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3458::DEBUG::2014-10-15 06:40:58,088::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2a546eb-3a2a-4fd0-b3a2-b66fbe6d99ba`::moving from state init -> state preparing >Thread-3458::INFO::2014-10-15 06:40:58,088::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3458::INFO::2014-10-15 06:40:58,101::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3458::DEBUG::2014-10-15 06:40:58,101::task::1191::Storage.TaskManager.Task::(prepare) Task=`e2a546eb-3a2a-4fd0-b3a2-b66fbe6d99ba`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3458::DEBUG::2014-10-15 06:40:58,101::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2a546eb-3a2a-4fd0-b3a2-b66fbe6d99ba`::moving from state preparing -> state finished >Thread-3458::DEBUG::2014-10-15 06:40:58,101::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3458::DEBUG::2014-10-15 06:40:58,102::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3458::DEBUG::2014-10-15 06:40:58,102::task::993::Storage.TaskManager.Task::(_decref) Task=`e2a546eb-3a2a-4fd0-b3a2-b66fbe6d99ba`::ref 0 aborting False >Thread-3458::DEBUG::2014-10-15 06:40:58,102::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3458::DEBUG::2014-10-15 06:40:58,102::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:40:58,109::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:40:58,110::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3459::DEBUG::2014-10-15 06:40:58,110::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3459::DEBUG::2014-10-15 06:40:58,111::task::595::Storage.TaskManager.Task::(_updateState) Task=`91d4e18d-7b76-4dcb-97a9-734fe1ffd566`::moving from state init -> state preparing >Thread-3459::INFO::2014-10-15 06:40:58,111::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3459::DEBUG::2014-10-15 06:40:58,111::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`c071435c-baff-4c1f-a220-c316f1394125`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3459::DEBUG::2014-10-15 06:40:58,111::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3459::DEBUG::2014-10-15 06:40:58,112::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3459::DEBUG::2014-10-15 06:40:58,112::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`c071435c-baff-4c1f-a220-c316f1394125`::Granted request >Thread-3459::DEBUG::2014-10-15 06:40:58,112::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`91d4e18d-7b76-4dcb-97a9-734fe1ffd566`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3459::DEBUG::2014-10-15 06:40:58,112::task::993::Storage.TaskManager.Task::(_decref) Task=`91d4e18d-7b76-4dcb-97a9-734fe1ffd566`::ref 1 aborting False >Thread-3459::INFO::2014-10-15 06:40:58,130::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3459::DEBUG::2014-10-15 06:40:58,130::task::1191::Storage.TaskManager.Task::(prepare) Task=`91d4e18d-7b76-4dcb-97a9-734fe1ffd566`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3459::DEBUG::2014-10-15 06:40:58,130::task::595::Storage.TaskManager.Task::(_updateState) Task=`91d4e18d-7b76-4dcb-97a9-734fe1ffd566`::moving from state preparing -> state finished >Thread-3459::DEBUG::2014-10-15 06:40:58,130::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3459::DEBUG::2014-10-15 06:40:58,130::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3459::DEBUG::2014-10-15 06:40:58,131::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3459::DEBUG::2014-10-15 06:40:58,131::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3459::DEBUG::2014-10-15 06:40:58,131::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3459::DEBUG::2014-10-15 06:40:58,131::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3459::DEBUG::2014-10-15 06:40:58,131::task::993::Storage.TaskManager.Task::(_decref) Task=`91d4e18d-7b76-4dcb-97a9-734fe1ffd566`::ref 0 aborting False >Thread-3459::DEBUG::2014-10-15 06:40:58,131::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3459::DEBUG::2014-10-15 06:40:58,131::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:40:58,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:40:58,979::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141408 s, 72.4 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:41:00,298::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:41:00,299::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:41:00,299::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:00,696::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:00,697::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3460::DEBUG::2014-10-15 06:41:00,697::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:00,699::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:00,699::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3461::DEBUG::2014-10-15 06:41:00,700::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3461::DEBUG::2014-10-15 06:41:00,700::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:41:00,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:41:01,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115587 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-838::DEBUG::2014-10-15 06:41:03,014::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:41:03,037::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178989 s, 57.2 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:41:03,284::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:41:03,284::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:41:03,284::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:41:03,284::__init__::232::IOProcess::(_processLogs) (2543) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:41:03,284::__init__::232::IOProcess::(_processLogs) (2543) Queuing response >Thread-24::DEBUG::2014-10-15 06:41:03,285::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:41:03,290::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000409824 s, 1.1 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:41:03,291::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:41:03,291::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:41:03,291::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:41:03,291::__init__::232::IOProcess::(_processLogs) (2544) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:41:03,292::__init__::232::IOProcess::(_processLogs) (2544) Queuing response >Thread-21::DEBUG::2014-10-15 06:41:03,292::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:41:03,292::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:41:03,292::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:41:03,292::__init__::232::IOProcess::(_processLogs) (2545) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:41:03,292::__init__::232::IOProcess::(_processLogs) (2545) Queuing response >Thread-21::DEBUG::2014-10-15 06:41:03,293::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:41:03,293::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:41:03,293::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:41:03,293::__init__::232::IOProcess::(_processLogs) (2546) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:41:03,293::__init__::232::IOProcess::(_processLogs) (2546) Queuing response >Thread-21::DEBUG::2014-10-15 06:41:03,293::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:41:03,294::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:41:03,294::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:41:03,294::__init__::232::IOProcess::(_processLogs) (2547) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:41:03,294::__init__::232::IOProcess::(_processLogs) (2547) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:03,740::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:03,740::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3462::DEBUG::2014-10-15 06:41:03,741::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:03,743::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:03,743::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3463::DEBUG::2014-10-15 06:41:03,744::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3463::DEBUG::2014-10-15 06:41:03,744::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:41:05,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:41:05,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163841 s, 62.5 MB/s\n'; <rc> = 0 >Thread-167::DEBUG::2014-10-15 06:41:05,219::task::595::Storage.TaskManager.Task::(_updateState) Task=`73811c3a-556f-45df-80b2-4602acd4bd84`::moving from state init -> state preparing >Thread-167::INFO::2014-10-15 06:41:05,220::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-21::DEBUG::2014-10-15 06:41:05,220::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:41:05,220::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:41:05,220::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:41:05,220::__init__::232::IOProcess::(_processLogs) (2548) Got request for method 'stat' >Thread-21::DEBUG::2014-10-15 06:41:05,221::__init__::232::IOProcess::(_processLogs) (2548) Queuing response >Thread-21::DEBUG::2014-10-15 06:41:05,222::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:41:05,222::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:41:05,222::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:41:05,222::__init__::232::IOProcess::(_processLogs) (2549) Got request for method 'stat' >Thread-21::DEBUG::2014-10-15 06:41:05,222::__init__::232::IOProcess::(_processLogs) (2549) Queuing response >Thread-167::INFO::2014-10-15 06:41:05,222::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-167::DEBUG::2014-10-15 06:41:05,222::task::1191::Storage.TaskManager.Task::(prepare) Task=`73811c3a-556f-45df-80b2-4602acd4bd84`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-167::DEBUG::2014-10-15 06:41:05,223::task::595::Storage.TaskManager.Task::(_updateState) Task=`73811c3a-556f-45df-80b2-4602acd4bd84`::moving from state preparing -> state finished >Thread-167::DEBUG::2014-10-15 06:41:05,223::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-167::DEBUG::2014-10-15 06:41:05,223::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-167::DEBUG::2014-10-15 06:41:05,223::task::993::Storage.TaskManager.Task::(_decref) Task=`73811c3a-556f-45df-80b2-4602acd4bd84`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:41:05,305::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:41:05,305::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:41:05,306::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:06,765::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:06,766::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3464::DEBUG::2014-10-15 06:41:06,767::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:06,768::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:06,768::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3465::DEBUG::2014-10-15 06:41:06,769::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3465::DEBUG::2014-10-15 06:41:06,769::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:41:07,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:41:07,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179637 s, 57.0 MB/s\n'; <rc> = 0 >VM Channels Listener::DEBUG::2014-10-15 06:41:07,483::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 47. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:08,148::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:08,149::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3466::DEBUG::2014-10-15 06:41:08,149::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3466::DEBUG::2014-10-15 06:41:08,150::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb7b1474-a548-4ecd-8140-93fd3196a2a1`::moving from state init -> state preparing >Thread-3466::INFO::2014-10-15 06:41:08,150::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3466::INFO::2014-10-15 06:41:08,162::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3466::DEBUG::2014-10-15 06:41:08,163::task::1191::Storage.TaskManager.Task::(prepare) Task=`eb7b1474-a548-4ecd-8140-93fd3196a2a1`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3466::DEBUG::2014-10-15 06:41:08,163::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb7b1474-a548-4ecd-8140-93fd3196a2a1`::moving from state preparing -> state finished >Thread-3466::DEBUG::2014-10-15 06:41:08,163::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3466::DEBUG::2014-10-15 06:41:08,163::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3466::DEBUG::2014-10-15 06:41:08,163::task::993::Storage.TaskManager.Task::(_decref) Task=`eb7b1474-a548-4ecd-8140-93fd3196a2a1`::ref 0 aborting False >Thread-3466::DEBUG::2014-10-15 06:41:08,163::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3466::DEBUG::2014-10-15 06:41:08,163::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:08,170::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-3467::DEBUG::2014-10-15 06:41:08,171::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >JsonRpcServer::DEBUG::2014-10-15 06:41:08,171::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3467::DEBUG::2014-10-15 06:41:08,171::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d80066b-29cb-4a87-9323-52b2d9d04ea3`::moving from state init -> state preparing >Thread-3467::INFO::2014-10-15 06:41:08,172::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3467::DEBUG::2014-10-15 06:41:08,172::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`cbb83ab9-97a2-4b02-b18b-4b5a9d17a4a0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3467::DEBUG::2014-10-15 06:41:08,172::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3467::DEBUG::2014-10-15 06:41:08,172::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3467::DEBUG::2014-10-15 06:41:08,172::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`cbb83ab9-97a2-4b02-b18b-4b5a9d17a4a0`::Granted request >Thread-3467::DEBUG::2014-10-15 06:41:08,172::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9d80066b-29cb-4a87-9323-52b2d9d04ea3`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3467::DEBUG::2014-10-15 06:41:08,173::task::993::Storage.TaskManager.Task::(_decref) Task=`9d80066b-29cb-4a87-9323-52b2d9d04ea3`::ref 1 aborting False >Thread-3467::INFO::2014-10-15 06:41:08,190::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3467::DEBUG::2014-10-15 06:41:08,190::task::1191::Storage.TaskManager.Task::(prepare) Task=`9d80066b-29cb-4a87-9323-52b2d9d04ea3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3467::DEBUG::2014-10-15 06:41:08,190::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d80066b-29cb-4a87-9323-52b2d9d04ea3`::moving from state preparing -> state finished >Thread-3467::DEBUG::2014-10-15 06:41:08,190::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3467::DEBUG::2014-10-15 06:41:08,191::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3467::DEBUG::2014-10-15 06:41:08,191::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3467::DEBUG::2014-10-15 06:41:08,191::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3467::DEBUG::2014-10-15 06:41:08,191::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3467::DEBUG::2014-10-15 06:41:08,191::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3467::DEBUG::2014-10-15 06:41:08,191::task::993::Storage.TaskManager.Task::(_decref) Task=`9d80066b-29cb-4a87-9323-52b2d9d04ea3`::ref 0 aborting False >Thread-3467::DEBUG::2014-10-15 06:41:08,191::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3467::DEBUG::2014-10-15 06:41:08,191::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:41:09,108::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:41:09,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0111812 s, 91.6 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:09,791::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:09,792::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3468::DEBUG::2014-10-15 06:41:09,793::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:09,794::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:09,795::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3469::DEBUG::2014-10-15 06:41:09,796::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3469::DEBUG::2014-10-15 06:41:09,796::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:41:10,311::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:41:10,311::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:41:10,312::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:41:11,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:41:11,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114463 s, 89.5 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:12,825::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:12,826::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3470::DEBUG::2014-10-15 06:41:12,828::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9b4bc67-01e8-4c58-9d70-c0fb12b774ef`::moving from state init -> state preparing >Thread-3470::INFO::2014-10-15 06:41:12,828::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-3470::INFO::2014-10-15 06:41:12,828::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000409824', 'lastCheck': '9.5', 'valid': True}} >Thread-3470::DEBUG::2014-10-15 06:41:12,828::task::1191::Storage.TaskManager.Task::(prepare) Task=`b9b4bc67-01e8-4c58-9d70-c0fb12b774ef`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000409824', 'lastCheck': '9.5', 'valid': True}} >Thread-3470::DEBUG::2014-10-15 06:41:12,828::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9b4bc67-01e8-4c58-9d70-c0fb12b774ef`::moving from state preparing -> state finished >Thread-3470::DEBUG::2014-10-15 06:41:12,828::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3470::DEBUG::2014-10-15 06:41:12,828::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3470::DEBUG::2014-10-15 06:41:12,828::task::993::Storage.TaskManager.Task::(_decref) Task=`b9b4bc67-01e8-4c58-9d70-c0fb12b774ef`::ref 0 aborting False >Thread-3470::ERROR::2014-10-15 06:41:12,829::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-3470::DEBUG::2014-10-15 06:41:12,830::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:12,841::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:12,842::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3471::DEBUG::2014-10-15 06:41:12,843::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3471::DEBUG::2014-10-15 06:41:12,843::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:41:13,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:41:13,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114619 s, 89.3 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:41:13,295::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:41:13,295::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:41:13,295::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:41:13,295::__init__::232::IOProcess::(_processLogs) (2550) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:41:13,295::__init__::232::IOProcess::(_processLogs) (2550) Queuing response >Thread-24::DEBUG::2014-10-15 06:41:13,296::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:41:13,301::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000384616 s, 1.2 MB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-10-15 06:41:13,302::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:41:13,302::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:41:13,302::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:41:13,302::__init__::232::IOProcess::(_processLogs) (2551) Got request for method 'statvfs' >Thread-21::DEBUG::2014-10-15 06:41:13,303::__init__::232::IOProcess::(_processLogs) (2551) Queuing response >Thread-21::DEBUG::2014-10-15 06:41:13,303::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:41:13,303::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:41:13,303::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:41:13,303::__init__::232::IOProcess::(_processLogs) (2552) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:41:13,304::__init__::232::IOProcess::(_processLogs) (2552) Queuing response >Thread-21::DEBUG::2014-10-15 06:41:13,304::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:41:13,304::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:41:13,304::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:41:13,304::__init__::232::IOProcess::(_processLogs) (2553) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:41:13,304::__init__::232::IOProcess::(_processLogs) (2553) Queuing response >Thread-21::DEBUG::2014-10-15 06:41:13,305::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-21::DEBUG::2014-10-15 06:41:13,305::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-21::DEBUG::2014-10-15 06:41:13,305::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-21::DEBUG::2014-10-15 06:41:13,305::__init__::232::IOProcess::(_processLogs) (2554) Got request for method 'access' >Thread-21::DEBUG::2014-10-15 06:41:13,305::__init__::232::IOProcess::(_processLogs) (2554) Queuing response >Dummy-838::DEBUG::2014-10-15 06:41:15,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:41:15,203::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114741 s, 89.2 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:41:15,318::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:41:15,318::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:41:15,319::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:15,935::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:15,936::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3472::DEBUG::2014-10-15 06:41:15,937::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:15,938::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:15,939::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3473::DEBUG::2014-10-15 06:41:15,939::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3473::DEBUG::2014-10-15 06:41:15,940::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:41:17,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:41:17,232::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0143945 s, 71.1 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:18,207::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:18,208::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3474::DEBUG::2014-10-15 06:41:18,208::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-3474::DEBUG::2014-10-15 06:41:18,209::task::595::Storage.TaskManager.Task::(_updateState) Task=`be72aaa4-7b9f-470d-a384-7cca8221bb0b`::moving from state init -> state preparing >Thread-3474::INFO::2014-10-15 06:41:18,209::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3474::INFO::2014-10-15 06:41:18,222::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3474::DEBUG::2014-10-15 06:41:18,222::task::1191::Storage.TaskManager.Task::(prepare) Task=`be72aaa4-7b9f-470d-a384-7cca8221bb0b`::finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L}} >Thread-3474::DEBUG::2014-10-15 06:41:18,222::task::595::Storage.TaskManager.Task::(_updateState) Task=`be72aaa4-7b9f-470d-a384-7cca8221bb0b`::moving from state preparing -> state finished >Thread-3474::DEBUG::2014-10-15 06:41:18,222::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-3474::DEBUG::2014-10-15 06:41:18,223::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3474::DEBUG::2014-10-15 06:41:18,223::task::993::Storage.TaskManager.Task::(_decref) Task=`be72aaa4-7b9f-470d-a384-7cca8221bb0b`::ref 0 aborting False >Thread-3474::DEBUG::2014-10-15 06:41:18,223::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18L} >Thread-3474::DEBUG::2014-10-15 06:41:18,223::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:18,231::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-3475::DEBUG::2014-10-15 06:41:18,231::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >JsonRpcServer::DEBUG::2014-10-15 06:41:18,231::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3475::DEBUG::2014-10-15 06:41:18,232::task::595::Storage.TaskManager.Task::(_updateState) Task=`5660d92c-c8a6-4961-83d3-eaaa0240c41f`::moving from state init -> state preparing >Thread-3475::INFO::2014-10-15 06:41:18,232::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-3475::DEBUG::2014-10-15 06:41:18,233::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`918f6da5-25ce-45dc-8550-09c36284798f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-3475::DEBUG::2014-10-15 06:41:18,233::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'shared' >Thread-3475::DEBUG::2014-10-15 06:41:18,233::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'shared' (1 active user) >Thread-3475::DEBUG::2014-10-15 06:41:18,233::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`918f6da5-25ce-45dc-8550-09c36284798f`::Granted request >Thread-3475::DEBUG::2014-10-15 06:41:18,233::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5660d92c-c8a6-4961-83d3-eaaa0240c41f`::_resourcesAcquired: Storage.00000002-0002-0002-0002-0000000003e1 (shared) >Thread-3475::DEBUG::2014-10-15 06:41:18,234::task::993::Storage.TaskManager.Task::(_decref) Task=`5660d92c-c8a6-4961-83d3-eaaa0240c41f`::ref 1 aborting False >Thread-3475::INFO::2014-10-15 06:41:18,252::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3475::DEBUG::2014-10-15 06:41:18,252::task::1191::Storage.TaskManager.Task::(prepare) Task=`5660d92c-c8a6-4961-83d3-eaaa0240c41f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3475::DEBUG::2014-10-15 06:41:18,252::task::595::Storage.TaskManager.Task::(_updateState) Task=`5660d92c-c8a6-4961-83d3-eaaa0240c41f`::moving from state preparing -> state finished >Thread-3475::DEBUG::2014-10-15 06:41:18,252::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-0000000003e1': < ResourceRef 'Storage.00000002-0002-0002-0002-0000000003e1', isValid: 'True' obj: 'None'>} >Thread-3475::DEBUG::2014-10-15 06:41:18,252::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-3475::DEBUG::2014-10-15 06:41:18,252::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-3475::DEBUG::2014-10-15 06:41:18,252::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-3475::DEBUG::2014-10-15 06:41:18,252::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-3475::DEBUG::2014-10-15 06:41:18,253::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-3475::DEBUG::2014-10-15 06:41:18,253::task::993::Storage.TaskManager.Task::(_decref) Task=`5660d92c-c8a6-4961-83d3-eaaa0240c41f`::ref 0 aborting False >Thread-3475::DEBUG::2014-10-15 06:41:18,253::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 18L, 'domains': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'master_uuid': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'version': '3', 'spm_id': 2, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'status': u'Active', 'diskfree': '261809504256', 'isoprefix': '', 'alerts': [], 'disktotal': '430753972224', 'version': 3}}} >Thread-3475::DEBUG::2014-10-15 06:41:18,253::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:18,970::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:18,971::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3476::DEBUG::2014-10-15 06:41:18,972::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:18,973::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:18,973::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-3477::DEBUG::2014-10-15 06:41:18,974::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-3477::DEBUG::2014-10-15 06:41:18,975::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-838::DEBUG::2014-10-15 06:41:19,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:41:19,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114738 s, 89.2 MB/s\n'; <rc> = 0 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:41:20,325::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:41:20,325::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:41:20,326::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Dummy-838::DEBUG::2014-10-15 06:41:21,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-838::DEBUG::2014-10-15 06:41:21,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0114578 s, 89.4 MB/s\n'; <rc> = 0 >MainThread::DEBUG::2014-10-15 06:41:21,830::vdsm::58::vds::(sigtermHandler) Received signal 15 >MainThread::DEBUG::2014-10-15 06:41:21,830::protocoldetector::135::vds.MultiProtocolAcceptor::(stop) Stopping Acceptor >MainThread::INFO::2014-10-15 06:41:21,831::__init__::563::jsonrpc.JsonRpcServer::(stop) Stopping JsonRPC Server >Thread-21::ERROR::2014-10-15 06:41:21,831::__init__::317::IOProcessClient::(_communicate) IOProcess failure >Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 272, in _communicate >Exception: FD closed >Detector thread::DEBUG::2014-10-15 06:41:21,832::protocoldetector::106::vds.MultiProtocolAcceptor::(_cleanup) Cleaning Acceptor >MainThread::INFO::2014-10-15 06:41:21,833::vmchannels::188::vds::(stop) VM channels listener was stopped. >Thread-21::DEBUG::2014-10-15 06:41:21,833::__init__::159::IOProcessClient::(_run) Starting IOProcess... >MainThread::INFO::2014-10-15 06:41:21,833::momIF::91::MOM::(stop) Shutting down MOM >MainThread::DEBUG::2014-10-15 06:41:21,839::task::595::Storage.TaskManager.Task::(_updateState) Task=`807bc6ff-c96d-432b-ae1e-dc146431382d`::moving from state init -> state preparing >MainThread::INFO::2014-10-15 06:41:21,839::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2014-10-15 06:41:21,839::storageServer::706::Storage.ConnectionMonitor::(_monitorConnections) Monitoring stopped >Thread-3479::DEBUG::2014-10-15 06:41:21,850::__init__::232::IOProcess::(_processLogs) Closing unrelated FDs... >Thread-3479::DEBUG::2014-10-15 06:41:21,850::__init__::232::IOProcess::(_processLogs) Opening communication channels... >VM Channels Listener::INFO::2014-10-15 06:41:22,498::vmchannels::183::vds::(run) VM channels listener thread has ended. >Dummy-838::INFO::2014-10-15 06:41:23,293::storage_mailbox::792::Storage.MailBox.SpmMailMonitor::(run) SPM_MailMonitor - Incoming mail monitoring thread stopped >Thread-3479::DEBUG::2014-10-15 06:41:23,306::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-3479::DEBUG::2014-10-15 06:41:23,306::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-3479::DEBUG::2014-10-15 06:41:23,307::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-3479::DEBUG::2014-10-15 06:41:23,307::__init__::232::IOProcess::(_processLogs) (2555) Got request for method 'statvfs' >Thread-3479::DEBUG::2014-10-15 06:41:23,308::__init__::232::IOProcess::(_processLogs) (2555) Queuing response >Thread-24::DEBUG::2014-10-15 06:41:23,308::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-24::DEBUG::2014-10-15 06:41:23,315::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000469677 s, 977 kB/s\n'; <rc> = 0 >Thread-3479::DEBUG::2014-10-15 06:41:23,316::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-3479::DEBUG::2014-10-15 06:41:23,316::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-3479::DEBUG::2014-10-15 06:41:23,316::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-3479::DEBUG::2014-10-15 06:41:23,316::__init__::232::IOProcess::(_processLogs) (2556) Got request for method 'statvfs' >Thread-3479::DEBUG::2014-10-15 06:41:23,316::__init__::232::IOProcess::(_processLogs) (2556) Queuing response >Thread-3479::DEBUG::2014-10-15 06:41:23,317::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-3479::DEBUG::2014-10-15 06:41:23,317::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-3479::DEBUG::2014-10-15 06:41:23,317::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-3479::DEBUG::2014-10-15 06:41:23,317::__init__::232::IOProcess::(_processLogs) (2557) Got request for method 'access' >Thread-3479::DEBUG::2014-10-15 06:41:23,317::__init__::232::IOProcess::(_processLogs) (2557) Queuing response >Thread-3479::DEBUG::2014-10-15 06:41:23,318::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-3479::DEBUG::2014-10-15 06:41:23,318::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-3479::DEBUG::2014-10-15 06:41:23,318::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-3479::DEBUG::2014-10-15 06:41:23,318::__init__::232::IOProcess::(_processLogs) (2558) Got request for method 'access' >Thread-3479::DEBUG::2014-10-15 06:41:23,318::__init__::232::IOProcess::(_processLogs) (2558) Queuing response >Thread-3479::DEBUG::2014-10-15 06:41:23,318::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-3479::DEBUG::2014-10-15 06:41:23,318::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-3479::DEBUG::2014-10-15 06:41:23,318::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-3479::DEBUG::2014-10-15 06:41:23,319::__init__::232::IOProcess::(_processLogs) (2559) Got request for method 'access' >Thread-3479::DEBUG::2014-10-15 06:41:23,319::__init__::232::IOProcess::(_processLogs) (2559) Queuing response >MainThread::INFO::2014-10-15 06:41:23,623::domainMonitor::137::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::INFO::2014-10-15 06:41:23,624::domainMonitor::114::Storage.DomainMonitor::(stopMonitoring) Stop monitoring 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-24::DEBUG::2014-10-15 06:41:23,624::domainMonitor::192::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-24::INFO::2014-10-15 06:41:23,624::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 (id: 2) >Thread-24::DEBUG::2014-10-15 06:41:23,624::domainMonitor::201::Storage.DomainMonitorThread::(_monitorLoop) Unable to release the host id 2 for domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 198, in _monitorLoop > File "/usr/share/vdsm/storage/sd.py", line 480, in releaseHostId > File "/usr/share/vdsm/storage/clusterlock.py", line 252, in releaseHostId >ReleaseHostIdFailure: Cannot release host id: (u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', SanlockException(16, 'Sanlock lockspace remove failure', 'Device or resource busy')) >MainThread::DEBUG::2014-10-15 06:41:23,624::taskManager::90::Storage.TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2014-10-15 06:41:23,624::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2014-10-15 06:41:23,625::task::1191::Storage.TaskManager.Task::(prepare) Task=`807bc6ff-c96d-432b-ae1e-dc146431382d`::finished: None >MainThread::DEBUG::2014-10-15 06:41:23,625::task::595::Storage.TaskManager.Task::(_updateState) Task=`807bc6ff-c96d-432b-ae1e-dc146431382d`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-15 06:41:23,625::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-15 06:41:23,625::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-15 06:41:23,625::task::993::Storage.TaskManager.Task::(_decref) Task=`807bc6ff-c96d-432b-ae1e-dc146431382d`::ref 0 aborting False >MainThread::INFO::2014-10-15 06:41:23,625::vdsm::137::vds::(run) VDSM main thread ended. Waiting for 19 other threads... >MainThread::INFO::2014-10-15 06:41:23,625::vdsm::141::vds::(run) <Thread(libvirtEventLoop, started daemon 140453635163904)> >MainThread::INFO::2014-10-15 06:41:23,625::vdsm::141::vds::(run) <WorkerThread(Thread-6, started daemon 140453517178624)> >MainThread::INFO::2014-10-15 06:41:23,625::vdsm::141::vds::(run) <HSM_MailMonitor(Thread-33, started daemon 140451688453888)> >MainThread::INFO::2014-10-15 06:41:23,625::vdsm::141::vds::(run) <GuestMonitor(GuestMonitor-rhevh-guest-rhel7.0, started daemon 140451201939200)> >MainThread::INFO::2014-10-15 06:41:23,625::vdsm::141::vds::(run) <WorkerThread(Thread-1, started daemon 140453626771200)> >MainThread::INFO::2014-10-15 06:41:23,626::vdsm::141::vds::(run) <WorkerThread(Thread-8, started daemon 140453500393216)> >MainThread::INFO::2014-10-15 06:41:23,626::vdsm::141::vds::(run) <WorkerThread(Thread-7, started daemon 140453508785920)> >MainThread::INFO::2014-10-15 06:41:23,626::vdsm::141::vds::(run) <WorkerThread(Thread-5, started daemon 140453525571328)> >MainThread::INFO::2014-10-15 06:41:23,626::vdsm::141::vds::(run) <WorkerThread(Thread-4, started daemon 140453533964032)> >MainThread::INFO::2014-10-15 06:41:23,626::vdsm::141::vds::(run) <WorkerThread(Thread-10, started daemon 140452938376960)> >MainThread::INFO::2014-10-15 06:41:23,626::vdsm::141::vds::(run) <WorkerThread(Thread-9, started daemon 140452946769664)> >MainThread::INFO::2014-10-15 06:41:23,626::vdsm::141::vds::(run) <PolicyEngine(PolicyEngine, started daemon 140452300826368)> >MainThread::DEBUG::2014-10-15 06:41:23,626::sampling::412::vm.Vm::(stop) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stop statistics collection >Thread-167::DEBUG::2014-10-15 06:41:23,626::sampling::443::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stats thread finished >MainThread::INFO::2014-10-15 06:41:23,626::vdsm::141::vds::(run) <VmStatsThread(Thread-167, started daemon 140452292433664)> >MainThread::INFO::2014-10-15 06:41:23,627::vdsm::141::vds::(run) <_MainThread(MainThread, started 140453871847232)> >MainThread::INFO::2014-10-15 06:41:23,627::vdsm::141::vds::(run) <WorkerThread(92171b82-b4a2-467f-a1bb-45c6554c2941, started daemon 140453550749440)> >MainThread::INFO::2014-10-15 06:41:23,627::vdsm::141::vds::(run) <Thread(Thread-3479, started daemon 140451134797568)> >MainThread::INFO::2014-10-15 06:41:23,627::vdsm::141::vds::(run) <_DummyThread(Dummy-838, started daemon 140451151582976)> >MainThread::INFO::2014-10-15 06:41:23,627::vdsm::141::vds::(run) <WorkerThread(Thread-3, started daemon 140453542356736)> >MainThread::INFO::2014-10-15 06:41:23,627::momIF::91::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-10-15 06:41:23,627::vdsm::141::vds::(run) <MomThread(MOM, started 140452653635328)> >MainThread::INFO::2014-10-15 06:41:23,627::vdsm::141::vds::(run) <GuestManager(GuestManager, started daemon 140452326004480)> >Thread-33::INFO::2014-10-15 06:41:23,766::storage_mailbox::509::Storage.MailBox.HsmMailMonitor::(run) HSM_MailboxMonitor - Incoming mail monitoring thread stopped, clearing outgoing mail >Thread-33::INFO::2014-10-15 06:41:23,766::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/usr/bin/dd', 'of=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=16'] >Thread-33::DEBUG::2014-10-15 06:41:23,767::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /usr/bin/dd of=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=16 (cwd None) >Thread-33::DEBUG::2014-10-15 06:41:23,876::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.105967 s, 38.7 kB/s\n'; <rc> = 0 >MainThread::INFO::2014-10-15 06:41:35,243::vdsm::131::vds::(run) (PID: 14460) I am the actual vdsm 4.16.6-1.el7 hp-z220-04.qe.lab.eng.nay.redhat.com (3.10.0-123.8.1.el7.x86_64) >MainThread::DEBUG::2014-10-15 06:41:35,244::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2014-10-15 06:41:35,244::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 10, waitTimeout: 3, maxTasks: 500 >MainThread::DEBUG::2014-10-15 06:41:35,246::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt >MainThread::WARNING::2014-10-15 06:41:35,246::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-10-15 06:41:35,263::supervdsm::77::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2014-10-15 06:41:35,267::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-10-15 06:41:35,267::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) /usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type (cwd None) >MainThread::DEBUG::2014-10-15 06:41:35,291::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-10-15 06:41:35,292::hsm::442::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-10-15 06:41:35,303::hsm::474::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-10-15 06:41:35,303::hsm::475::Storage.HSM::(__cleanStorageRepository) Mount list: ['/rhev/data-center', '/rhev/data-center/mnt/10.66.128.104:_home_NFS1'] >MainThread::DEBUG::2014-10-15 06:41:35,303::hsm::477::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-10-15 06:41:35,304::hsm::520::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2014-10-15 06:41:35,306::dispatcher::46::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-10-15 06:41:35,306::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >storageRefresh::DEBUG::2014-10-15 06:41:35,342::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-15 06:41:35,342::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >storageRefresh::DEBUG::2014-10-15 06:41:35,342::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-15 06:41:35,343::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >MainThread::DEBUG::2014-10-15 06:41:35,347::task::595::Storage.TaskManager.Task::(_updateState) Task=`822ecf6c-47ac-49ba-b9fc-c3546b7906a5`::moving from state init -> state preparing >storageRefresh::DEBUG::2014-10-15 06:41:35,347::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >MainThread::INFO::2014-10-15 06:41:35,347::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF object at 0x14ed750>>) >MainThread::INFO::2014-10-15 06:41:35,351::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::DEBUG::2014-10-15 06:41:35,351::task::1191::Storage.TaskManager.Task::(prepare) Task=`822ecf6c-47ac-49ba-b9fc-c3546b7906a5`::finished: None >MainThread::DEBUG::2014-10-15 06:41:35,351::task::595::Storage.TaskManager.Task::(_updateState) Task=`822ecf6c-47ac-49ba-b9fc-c3546b7906a5`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-15 06:41:35,351::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-15 06:41:35,351::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-15 06:41:35,352::task::993::Storage.TaskManager.Task::(_decref) Task=`822ecf6c-47ac-49ba-b9fc-c3546b7906a5`::ref 0 aborting False >MainThread::INFO::2014-10-15 06:41:35,352::momIF::49::MOM::(__init__) Starting up MOM >MainThread::INFO::2014-10-15 06:41:35,354::vmchannels::192::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2014-10-15 06:41:35,357::vmchannels::175::vds::(run) Starting VM channels listener thread. >storageRefresh::DEBUG::2014-10-15 06:41:35,357::misc::751::Storage.SamplingMethod::(__call__) Returning last result >clientIFinit::DEBUG::2014-10-15 06:41:35,364::libvirtconnection::150::root::(get) trying to connect libvirt >MainThread::DEBUG::2014-10-15 06:41:35,365::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <rpc.BindingXMLRPC.XmlDetector instance at 0x1b138c0> >storageRefresh::DEBUG::2014-10-15 06:41:35,386::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >storageRefresh::DEBUG::2014-10-15 06:41:35,427::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-15 06:41:35,430::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-15 06:41:35,431::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-15 06:41:35,432::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-15 06:41:35,432::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-15 06:41:35,432::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >MainThread::DEBUG::2014-10-15 06:41:35,433::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <yajsonrpc.stompReactor.StompDetector instance at 0x7fa8480eee18> >storageRefresh::DEBUG::2014-10-15 06:41:35,433::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-15 06:41:35,434::misc::751::Storage.SamplingMethod::(__call__) Returning last result >JsonRpcServer::DEBUG::2014-10-15 06:41:35,434::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >storageRefresh::DEBUG::2014-10-15 06:41:35,435::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >Detector thread::DEBUG::2014-10-15 06:41:35,436::protocoldetector::70::vds.MultiProtocolAcceptor::(serve_forever) Acceptor running >Detector thread::DEBUG::2014-10-15 06:41:35,444::protocoldetector::72::vds.MultiProtocolAcceptor::(serve_forever) Using required_size=7 >clientIFinit::DEBUG::2014-10-15 06:41:35,483::clientIF::494::vds::(_recoverVm) Trying to recover c255610f-42c2-426d-9270-e4497f4a0f37 >clientIFinit::INFO::2014-10-15 06:41:35,484::clientIF::390::vds::(createVm) vmContainerLock acquired by vm c255610f-42c2-426d-9270-e4497f4a0f37 >storageRefresh::DEBUG::2014-10-15 06:41:35,485::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size (cwd None) >clientIFinit::DEBUG::2014-10-15 06:41:35,494::libvirtconnection::150::root::(get) trying to connect libvirt >clientIFinit::DEBUG::2014-10-15 06:41:35,517::clientIF::403::vds::(createVm) Total desktops after creation of c255610f-42c2-426d-9270-e4497f4a0f37 is 1 >Thread-13::DEBUG::2014-10-15 06:41:35,517::vm::2259::vm.Vm::(_startUnderlyingVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Start >Thread-13::DEBUG::2014-10-15 06:41:35,519::vm::2263::vm.Vm::(_startUnderlyingVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::_ongoingCreations acquired >Thread-13::INFO::2014-10-15 06:41:35,519::vm::3299::vm.Vm::(_run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::VM wrapper has started >Thread-13::WARNING::2014-10-15 06:41:35,520::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': u'unix', 'alias': u'channel0', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'1'}}' found >Thread-13::WARNING::2014-10-15 06:41:35,521::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': u'unix', 'alias': u'channel1', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'2'}}' found >Thread-13::WARNING::2014-10-15 06:41:35,522::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': u'spicevmc', 'alias': u'channel2', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'3'}}' found >Thread-13::DEBUG::2014-10-15 06:41:35,523::task::595::Storage.TaskManager.Task::(_updateState) Task=`4169f37f-e865-48d0-820c-eacd216fa8da`::moving from state init -> state preparing >Thread-13::INFO::2014-10-15 06:41:35,524::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-13::ERROR::2014-10-15 06:41:35,524::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-13::ERROR::2014-10-15 06:41:35,525::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-13::DEBUG::2014-10-15 06:41:35,526::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >Thread-13::DEBUG::2014-10-15 06:41:35,526::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 (cwd None) >storageRefresh::DEBUG::2014-10-15 06:41:35,755::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-15 06:41:35,756::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >storageRefresh::DEBUG::2014-10-15 06:41:35,756::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-13::DEBUG::2014-10-15 06:41:35,784::lvm::288::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume group "55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4" not found\n Skipping volume group 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4\n'; <rc> = 5 >Thread-13::WARNING::2014-10-15 06:41:35,785::lvm::370::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4" not found', ' Skipping volume group 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4'] >Thread-15::DEBUG::2014-10-15 06:41:35,786::__init__::159::IOProcessClient::(_run) Starting IOProcess... >Thread-16::DEBUG::2014-10-15 06:41:35,791::__init__::232::IOProcess::(_processLogs) Closing unrelated FDs... >storageRefresh::DEBUG::2014-10-15 06:41:35,791::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:41:35,792::__init__::232::IOProcess::(_processLogs) Opening communication channels... >storageRefresh::DEBUG::2014-10-15 06:41:35,793::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-16::DEBUG::2014-10-15 06:41:35,793::__init__::232::IOProcess::(_processLogs) Receiving request... >storageRefresh::DEBUG::2014-10-15 06:41:35,794::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags (cwd None) >Thread-16::DEBUG::2014-10-15 06:41:35,794::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:41:35,799::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:41:35,800::__init__::232::IOProcess::(_processLogs) (1) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:41:35,800::__init__::232::IOProcess::(_processLogs) (1) Queuing response >Thread-13::DEBUG::2014-10-15 06:41:35,807::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 06:41:35,807::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:41:35,808::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:41:35,808::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:41:35,808::__init__::232::IOProcess::(_processLogs) (2) Got request for method 'touch' >storageRefresh::DEBUG::2014-10-15 06:41:35,825::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-15 06:41:35,826::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Data >storageRefresh::DEBUG::2014-10-15 06:41:35,826::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Logging >storageRefresh::DEBUG::2014-10-15 06:41:35,826::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Config >storageRefresh::DEBUG::2014-10-15 06:41:35,826::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Swap >storageRefresh::DEBUG::2014-10-15 06:41:35,826::hsm::387::Storage.HSM::(storageRefresh) HSM is ready >Thread-16::DEBUG::2014-10-15 06:41:35,847::__init__::232::IOProcess::(_processLogs) (2) Queuing response >Thread-13::DEBUG::2014-10-15 06:41:35,847::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-16::DEBUG::2014-10-15 06:41:35,847::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:41:35,848::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:41:35,848::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:41:35,848::__init__::232::IOProcess::(_processLogs) (3) Got request for method 'readfile' >Thread-16::DEBUG::2014-10-15 06:41:35,848::__init__::232::IOProcess::(_processLogs) (3) Queuing response >Thread-13::DEBUG::2014-10-15 06:41:35,849::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=domain1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-0000000003e1', 'REMOTE_PATH=10.66.128.104:/home/NFS1', 'ROLE=Master', 'SDUUID=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b90f33781d11f6e6b0b3945de3167c0719a458c1'] >Thread-16::DEBUG::2014-10-15 06:41:35,849::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:41:35,849::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:41:35,850::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:41:35,850::__init__::232::IOProcess::(_processLogs) (4) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:41:35,850::__init__::232::IOProcess::(_processLogs) (4) Queuing response >Thread-16::DEBUG::2014-10-15 06:41:35,850::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:41:35,850::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:41:35,850::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:41:35,850::__init__::232::IOProcess::(_processLogs) (5) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:41:35,850::__init__::232::IOProcess::(_processLogs) (5) Queuing response >Thread-13::DEBUG::2014-10-15 06:41:35,850::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-13::DEBUG::2014-10-15 06:41:35,851::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_imageNS' >Thread-13::DEBUG::2014-10-15 06:41:35,851::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_volumeNS' >Thread-16::DEBUG::2014-10-15 06:41:35,851::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:41:35,852::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:41:35,852::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:41:35,852::__init__::232::IOProcess::(_processLogs) (6) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:41:35,852::__init__::232::IOProcess::(_processLogs) (6) Queuing response >Thread-16::DEBUG::2014-10-15 06:41:35,852::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-13::INFO::2014-10-15 06:41:35,853::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-16::DEBUG::2014-10-15 06:41:35,853::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-13::DEBUG::2014-10-15 06:41:35,853::task::1191::Storage.TaskManager.Task::(prepare) Task=`4169f37f-e865-48d0-820c-eacd216fa8da`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-16::DEBUG::2014-10-15 06:41:35,853::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-13::DEBUG::2014-10-15 06:41:35,854::task::595::Storage.TaskManager.Task::(_updateState) Task=`4169f37f-e865-48d0-820c-eacd216fa8da`::moving from state preparing -> state finished >Thread-16::DEBUG::2014-10-15 06:41:35,854::__init__::232::IOProcess::(_processLogs) (7) Got request for method 'stat' >Thread-13::DEBUG::2014-10-15 06:41:35,854::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-10-15 06:41:35,855::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-16::DEBUG::2014-10-15 06:41:35,855::__init__::232::IOProcess::(_processLogs) (7) Queuing response >Thread-13::DEBUG::2014-10-15 06:41:35,855::task::993::Storage.TaskManager.Task::(_decref) Task=`4169f37f-e865-48d0-820c-eacd216fa8da`::ref 0 aborting False >Thread-13::DEBUG::2014-10-15 06:41:35,857::vm::1067::vm.Vm::(__init__) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Ignoring param (target, 1048576) in BalloonDevice >Thread-13::DEBUG::2014-10-15 06:41:35,872::vm::5301::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Looking for drive with attributes {'name': u'hdc', 'bootOrder': '', 'boot': [], 'readonly': True, 'address': {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, 'path': '', 'type': u'cdrom'} >Thread-13::DEBUG::2014-10-15 06:41:35,872::vm::5322::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'hdc', u'hdc'), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, True), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}), 'path': ('', ''), 'type': (u'cdrom', u'cdrom')} >Thread-13::DEBUG::2014-10-15 06:41:35,872::vm::5341::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'hdc', None), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, None), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, None), 'path': ('', None), 'type': (u'cdrom', None)} >Thread-13::DEBUG::2014-10-15 06:41:35,873::vm::5301::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Looking for drive with attributes {'name': u'vda', 'bootOrder': u'2', 'boot': [<DOM Element: boot at 0x7fa84413ca70>], 'readonly': False, 'address': {u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, 'path': u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'type': u'disk'} >Thread-13::DEBUG::2014-10-15 06:41:35,873::vm::5322::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'vda', u'vda'), 'bootOrder': (u'2', u'2'), 'boot': ([<DOM Element: boot at 0x7fa84413ca70>], None), 'readonly': (False, False), 'address': ({u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, {u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}), 'path': (u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb'), 'type': (u'disk', u'disk')} >Thread-13::DEBUG::2014-10-15 06:41:35,873::vm::5341::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'vda', None), 'bootOrder': (u'2', None), 'boot': ([<DOM Element: boot at 0x7fa84413ca70>], None), 'readonly': (False, None), 'address': ({u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, None), 'path': (u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', None), 'type': (u'disk', None)} >Thread-13::DEBUG::2014-10-15 06:41:35,908::sampling::405::vm.Vm::(start) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Start statistics collection >Thread-17::DEBUG::2014-10-15 06:41:35,908::sampling::434::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stats thread started >Thread-13::DEBUG::2014-10-15 06:41:35,910::vmchannels::199::vds::(register) Add fileno 12 to listener's channels. >Thread-13::WARNING::2014-10-15 06:41:35,911::vm::3971::vm.Vm::(_readPauseCode) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::_readPauseCode unsupported by libvirt vm >Thread-13::DEBUG::2014-10-15 06:41:35,915::vm::2289::vm.Vm::(_startUnderlyingVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::_ongoingCreations released >Thread-17::DEBUG::2014-10-15 06:41:35,917::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 74 edom: 10 level: 2 message: argument unsupported: QEMU driver does not support <metadata> element >Thread-17::ERROR::2014-10-15 06:41:35,917::vm::373::vm.Vm::(_sampleCpuTune) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::libvirt does not support metadata >VM Channels Listener::DEBUG::2014-10-15 06:41:36,358::vmchannels::107::vds::(_do_add_channels) fileno 12 was added to unconnected channels. >VM Channels Listener::DEBUG::2014-10-15 06:41:36,359::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 12. >VM Channels Listener::DEBUG::2014-10-15 06:41:36,359::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:41:36,359::guestagent::203::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Connected to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:41:36,359::vmchannels::149::vds::(_handle_unconnected) Connecting to fileno 12 succeeded. >clientIFinit::DEBUG::2014-10-15 06:41:36,520::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bf7884f-f948-4f7a-9d98-65b54af5abdb`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:41:36,520::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:41:36,520::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:41:36,520::task::1191::Storage.TaskManager.Task::(prepare) Task=`8bf7884f-f948-4f7a-9d98-65b54af5abdb`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:41:36,521::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bf7884f-f948-4f7a-9d98-65b54af5abdb`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:41:36,521::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:41:36,521::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:41:36,521::task::993::Storage.TaskManager.Task::(_decref) Task=`8bf7884f-f948-4f7a-9d98-65b54af5abdb`::ref 0 aborting False >Detector thread::DEBUG::2014-10-15 06:41:38,394::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.66.78.3:37583 >Detector thread::DEBUG::2014-10-15 06:41:38,394::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.66.78.3:37583 >Detector thread::DEBUG::2014-10-15 06:41:38,395::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol stomp from 10.66.78.3:37583 >Detector thread::DEBUG::2014-10-15 06:41:38,395::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='CONNECT'> >Detector thread::INFO::2014-10-15 06:41:38,395::stompReactor::66::Broker.StompAdapter::(_cmd_connect) Processing CONNECT request >Detector thread::DEBUG::2014-10-15 06:41:38,395::stompReactor::338::protocoldetector.StompDetector::(handleSocket) Stomp detected from ('10.66.78.3', 37583) >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:38,396::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SUBSCRIBE'> >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:38,396::stompReactor::89::Broker.StompAdapter::(_cmd_subscribe) Subscribe command ignored >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:38,436::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:41:38,436::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-18::DEBUG::2014-10-15 06:41:38,436::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1'} >Thread-18::DEBUG::2014-10-15 06:41:38,437::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e8b1014-2290-476e-a9b0-56cd28cb66bb`::moving from state init -> state preparing >Thread-18::INFO::2014-10-15 06:41:38,438::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-0000000003e1', options=None) >Thread-18::ERROR::2014-10-15 06:41:38,438::task::866::Storage.TaskManager.Task::(_setError) Task=`4e8b1014-2290-476e-a9b0-56cd28cb66bb`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 873, in _run > File "/usr/share/vdsm/logUtils.py", line 45, in wrapper > File "/usr/share/vdsm/storage/hsm.py", line 609, in getSpmStatus > File "/usr/share/vdsm/storage/hsm.py", line 325, in getPool >StoragePoolUnknown: Unknown pool id, pool not connected: (u'00000002-0002-0002-0002-0000000003e1',) >Thread-18::DEBUG::2014-10-15 06:41:38,438::task::885::Storage.TaskManager.Task::(_run) Task=`4e8b1014-2290-476e-a9b0-56cd28cb66bb`::Task._run: 4e8b1014-2290-476e-a9b0-56cd28cb66bb (u'00000002-0002-0002-0002-0000000003e1',) {} failed - stopping task >Thread-18::DEBUG::2014-10-15 06:41:38,438::task::1217::Storage.TaskManager.Task::(stop) Task=`4e8b1014-2290-476e-a9b0-56cd28cb66bb`::stopping in state preparing (force False) >Thread-18::DEBUG::2014-10-15 06:41:38,438::task::993::Storage.TaskManager.Task::(_decref) Task=`4e8b1014-2290-476e-a9b0-56cd28cb66bb`::ref 1 aborting True >Thread-18::INFO::2014-10-15 06:41:38,438::task::1171::Storage.TaskManager.Task::(prepare) Task=`4e8b1014-2290-476e-a9b0-56cd28cb66bb`::aborting: Task is aborted: 'Unknown pool id, pool not connected' - code 309 >Thread-18::DEBUG::2014-10-15 06:41:38,438::task::1176::Storage.TaskManager.Task::(prepare) Task=`4e8b1014-2290-476e-a9b0-56cd28cb66bb`::Prepare: aborted: Unknown pool id, pool not connected >Thread-18::DEBUG::2014-10-15 06:41:38,438::task::993::Storage.TaskManager.Task::(_decref) Task=`4e8b1014-2290-476e-a9b0-56cd28cb66bb`::ref 0 aborting True >Thread-18::DEBUG::2014-10-15 06:41:38,438::task::928::Storage.TaskManager.Task::(_doAbort) Task=`4e8b1014-2290-476e-a9b0-56cd28cb66bb`::Task._doAbort: force False >Thread-18::DEBUG::2014-10-15 06:41:38,439::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2014-10-15 06:41:38,439::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e8b1014-2290-476e-a9b0-56cd28cb66bb`::moving from state preparing -> state aborting >Thread-18::DEBUG::2014-10-15 06:41:38,439::task::550::Storage.TaskManager.Task::(__state_aborting) Task=`4e8b1014-2290-476e-a9b0-56cd28cb66bb`::_aborting: recover policy none >Thread-18::DEBUG::2014-10-15 06:41:38,439::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e8b1014-2290-476e-a9b0-56cd28cb66bb`::moving from state aborting -> state failed >Thread-18::DEBUG::2014-10-15 06:41:38,439::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2014-10-15 06:41:38,439::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::ERROR::2014-10-15 06:41:38,439::dispatcher::76::Storage.Dispatcher::(wrapper) {'status': {'message': "Unknown pool id, pool not connected: (u'00000002-0002-0002-0002-0000000003e1',)", 'code': 309}} >Thread-18::DEBUG::2014-10-15 06:41:38,439::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:41:38,440::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-19::DEBUG::2014-10-15 06:41:38,441::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getAllTasksStatuses' in bridge with {} >JsonRpcServer::DEBUG::2014-10-15 06:41:38,441::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-19::DEBUG::2014-10-15 06:41:38,442::task::595::Storage.TaskManager.Task::(_updateState) Task=`ab0f4a3d-a816-4042-b009-d586ad3e6f3a`::moving from state init -> state preparing >Thread-19::INFO::2014-10-15 06:41:38,442::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-19::ERROR::2014-10-15 06:41:38,442::task::866::Storage.TaskManager.Task::(_setError) Task=`ab0f4a3d-a816-4042-b009-d586ad3e6f3a`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 873, in _run > File "/usr/share/vdsm/logUtils.py", line 45, in wrapper > File "/usr/share/vdsm/storage/hsm.py", line 2168, in getAllTasksStatuses >SpmStatusError: Not SPM: () >Thread-19::DEBUG::2014-10-15 06:41:38,442::task::885::Storage.TaskManager.Task::(_run) Task=`ab0f4a3d-a816-4042-b009-d586ad3e6f3a`::Task._run: ab0f4a3d-a816-4042-b009-d586ad3e6f3a () {} failed - stopping task >Thread-19::DEBUG::2014-10-15 06:41:38,442::task::1217::Storage.TaskManager.Task::(stop) Task=`ab0f4a3d-a816-4042-b009-d586ad3e6f3a`::stopping in state preparing (force False) >Thread-19::DEBUG::2014-10-15 06:41:38,442::task::993::Storage.TaskManager.Task::(_decref) Task=`ab0f4a3d-a816-4042-b009-d586ad3e6f3a`::ref 1 aborting True >Thread-19::INFO::2014-10-15 06:41:38,442::task::1171::Storage.TaskManager.Task::(prepare) Task=`ab0f4a3d-a816-4042-b009-d586ad3e6f3a`::aborting: Task is aborted: 'Not SPM' - code 654 >Thread-19::DEBUG::2014-10-15 06:41:38,443::task::1176::Storage.TaskManager.Task::(prepare) Task=`ab0f4a3d-a816-4042-b009-d586ad3e6f3a`::Prepare: aborted: Not SPM >Thread-19::DEBUG::2014-10-15 06:41:38,443::task::993::Storage.TaskManager.Task::(_decref) Task=`ab0f4a3d-a816-4042-b009-d586ad3e6f3a`::ref 0 aborting True >Thread-19::DEBUG::2014-10-15 06:41:38,443::task::928::Storage.TaskManager.Task::(_doAbort) Task=`ab0f4a3d-a816-4042-b009-d586ad3e6f3a`::Task._doAbort: force False >Thread-19::DEBUG::2014-10-15 06:41:38,443::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-19::DEBUG::2014-10-15 06:41:38,443::task::595::Storage.TaskManager.Task::(_updateState) Task=`ab0f4a3d-a816-4042-b009-d586ad3e6f3a`::moving from state preparing -> state aborting >Thread-19::DEBUG::2014-10-15 06:41:38,443::task::550::Storage.TaskManager.Task::(__state_aborting) Task=`ab0f4a3d-a816-4042-b009-d586ad3e6f3a`::_aborting: recover policy none >Thread-19::DEBUG::2014-10-15 06:41:38,443::task::595::Storage.TaskManager.Task::(_updateState) Task=`ab0f4a3d-a816-4042-b009-d586ad3e6f3a`::moving from state aborting -> state failed >Thread-19::DEBUG::2014-10-15 06:41:38,443::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-19::DEBUG::2014-10-15 06:41:38,443::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-19::ERROR::2014-10-15 06:41:38,443::dispatcher::76::Storage.Dispatcher::(wrapper) {'status': {'message': 'Not SPM: ()', 'code': 654}} >Thread-19::DEBUG::2014-10-15 06:41:38,443::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >clientIFinit::DEBUG::2014-10-15 06:41:41,526::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e182848-d3f2-454d-9bcf-2d867a847c21`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:41:41,527::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:41:41,527::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:41:41,527::task::1191::Storage.TaskManager.Task::(prepare) Task=`5e182848-d3f2-454d-9bcf-2d867a847c21`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:41:41,527::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e182848-d3f2-454d-9bcf-2d867a847c21`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:41:41,527::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:41:41,527::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:41:41,527::task::993::Storage.TaskManager.Task::(_decref) Task=`5e182848-d3f2-454d-9bcf-2d867a847c21`::ref 0 aborting False >clientIFinit::DEBUG::2014-10-15 06:41:46,533::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c52d184-6826-4447-8625-f04125dce0e3`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:41:46,533::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:41:46,534::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:41:46,534::task::1191::Storage.TaskManager.Task::(prepare) Task=`5c52d184-6826-4447-8625-f04125dce0e3`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:41:46,534::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c52d184-6826-4447-8625-f04125dce0e3`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:41:46,534::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:41:46,534::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:41:46,534::task::993::Storage.TaskManager.Task::(_decref) Task=`5c52d184-6826-4447-8625-f04125dce0e3`::ref 0 aborting False >clientIFinit::DEBUG::2014-10-15 06:41:51,540::task::595::Storage.TaskManager.Task::(_updateState) Task=`fd4457c7-55a2-4b1d-a501-8c1061f51ef5`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:41:51,540::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:41:51,540::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:41:51,540::task::1191::Storage.TaskManager.Task::(prepare) Task=`fd4457c7-55a2-4b1d-a501-8c1061f51ef5`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:41:51,540::task::595::Storage.TaskManager.Task::(_updateState) Task=`fd4457c7-55a2-4b1d-a501-8c1061f51ef5`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:41:51,541::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:41:51,541::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:41:51,541::task::993::Storage.TaskManager.Task::(_decref) Task=`fd4457c7-55a2-4b1d-a501-8c1061f51ef5`::ref 0 aborting False >clientIFinit::DEBUG::2014-10-15 06:41:56,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd358b82-ef39-4461-908c-4295aaa8eba4`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:41:56,543::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:41:56,543::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:41:56,543::task::1191::Storage.TaskManager.Task::(prepare) Task=`dd358b82-ef39-4461-908c-4295aaa8eba4`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:41:56,543::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd358b82-ef39-4461-908c-4295aaa8eba4`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:41:56,543::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:41:56,543::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:41:56,543::task::993::Storage.TaskManager.Task::(_decref) Task=`dd358b82-ef39-4461-908c-4295aaa8eba4`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:00,458::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:00,459::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:00,459::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 06:42:01,546::task::595::Storage.TaskManager.Task::(_updateState) Task=`8f6ddb6b-88b5-4a79-9659-e6d4b6ed386a`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:42:01,546::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:42:01,547::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:01,547::task::1191::Storage.TaskManager.Task::(prepare) Task=`8f6ddb6b-88b5-4a79-9659-e6d4b6ed386a`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:01,547::task::595::Storage.TaskManager.Task::(_updateState) Task=`8f6ddb6b-88b5-4a79-9659-e6d4b6ed386a`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:42:01,547::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:42:01,547::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:42:01,547::task::993::Storage.TaskManager.Task::(_decref) Task=`8f6ddb6b-88b5-4a79-9659-e6d4b6ed386a`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:05,465::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:05,466::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:05,466::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:42:06,391::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 12. >clientIFinit::DEBUG::2014-10-15 06:42:06,553::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ad59a24-61a4-48cd-ab12-1041c259d803`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:42:06,553::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:42:06,553::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:06,554::task::1191::Storage.TaskManager.Task::(prepare) Task=`5ad59a24-61a4-48cd-ab12-1041c259d803`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:06,554::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ad59a24-61a4-48cd-ab12-1041c259d803`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:42:06,554::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:42:06,554::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:42:06,554::task::993::Storage.TaskManager.Task::(_decref) Task=`5ad59a24-61a4-48cd-ab12-1041c259d803`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:10,472::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:10,472::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:10,473::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 06:42:11,559::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d105246-0656-4c6f-977d-9df986304288`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:42:11,560::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:42:11,560::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:11,560::task::1191::Storage.TaskManager.Task::(prepare) Task=`3d105246-0656-4c6f-977d-9df986304288`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:11,560::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d105246-0656-4c6f-977d-9df986304288`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:42:11,560::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:42:11,560::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:42:11,561::task::993::Storage.TaskManager.Task::(_decref) Task=`3d105246-0656-4c6f-977d-9df986304288`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:15,478::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:15,479::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:15,479::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 06:42:16,566::task::595::Storage.TaskManager.Task::(_updateState) Task=`2fd5fb8c-dc86-43d1-abe8-d671dc657c4f`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:42:16,566::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:42:16,567::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:16,567::task::1191::Storage.TaskManager.Task::(prepare) Task=`2fd5fb8c-dc86-43d1-abe8-d671dc657c4f`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:16,567::task::595::Storage.TaskManager.Task::(_updateState) Task=`2fd5fb8c-dc86-43d1-abe8-d671dc657c4f`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:42:16,567::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:42:16,567::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:42:16,567::task::993::Storage.TaskManager.Task::(_decref) Task=`2fd5fb8c-dc86-43d1-abe8-d671dc657c4f`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:20,485::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:20,485::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:20,486::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 06:42:21,573::task::595::Storage.TaskManager.Task::(_updateState) Task=`d370bc7f-0964-4fa8-a4f9-a37dd5d9e8cc`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:42:21,573::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:42:21,573::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:21,574::task::1191::Storage.TaskManager.Task::(prepare) Task=`d370bc7f-0964-4fa8-a4f9-a37dd5d9e8cc`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:21,574::task::595::Storage.TaskManager.Task::(_updateState) Task=`d370bc7f-0964-4fa8-a4f9-a37dd5d9e8cc`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:42:21,574::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:42:21,574::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:42:21,574::task::993::Storage.TaskManager.Task::(_decref) Task=`d370bc7f-0964-4fa8-a4f9-a37dd5d9e8cc`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:25,492::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:25,492::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:42:25,493::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 06:42:26,579::task::595::Storage.TaskManager.Task::(_updateState) Task=`8150f0ec-590c-4633-9d43-9b74228018e2`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:42:26,580::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:42:26,580::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:26,580::task::1191::Storage.TaskManager.Task::(prepare) Task=`8150f0ec-590c-4633-9d43-9b74228018e2`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:26,580::task::595::Storage.TaskManager.Task::(_updateState) Task=`8150f0ec-590c-4633-9d43-9b74228018e2`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:42:26,580::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:42:26,581::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:42:26,581::task::993::Storage.TaskManager.Task::(_decref) Task=`8150f0ec-590c-4633-9d43-9b74228018e2`::ref 0 aborting False >MainThread::DEBUG::2014-10-15 06:42:30,173::vdsm::58::vds::(sigtermHandler) Received signal 15 >MainThread::DEBUG::2014-10-15 06:42:30,173::protocoldetector::135::vds.MultiProtocolAcceptor::(stop) Stopping Acceptor >MainThread::INFO::2014-10-15 06:42:30,173::__init__::563::jsonrpc.JsonRpcServer::(stop) Stopping JsonRPC Server >Detector thread::DEBUG::2014-10-15 06:42:30,173::protocoldetector::106::vds.MultiProtocolAcceptor::(_cleanup) Cleaning Acceptor >MainThread::INFO::2014-10-15 06:42:30,174::vmchannels::188::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-10-15 06:42:30,174::momIF::91::MOM::(stop) Shutting down MOM >MainThread::DEBUG::2014-10-15 06:42:30,174::task::595::Storage.TaskManager.Task::(_updateState) Task=`a565eb9a-b74e-49e2-97a0-0387d35ecd90`::moving from state init -> state preparing >Thread-16::ERROR::2014-10-15 06:42:30,174::__init__::317::IOProcessClient::(_communicate) IOProcess failure >Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 272, in _communicate >Exception: FD closed >MainThread::INFO::2014-10-15 06:42:30,175::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2014-10-15 06:42:30,176::storageServer::706::Storage.ConnectionMonitor::(_monitorConnections) Monitoring stopped >Thread-16::DEBUG::2014-10-15 06:42:30,178::__init__::159::IOProcessClient::(_run) Starting IOProcess... >MainThread::INFO::2014-10-15 06:42:30,194::domainMonitor::137::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::DEBUG::2014-10-15 06:42:30,195::taskManager::90::Storage.TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2014-10-15 06:42:30,195::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2014-10-15 06:42:30,195::task::1191::Storage.TaskManager.Task::(prepare) Task=`a565eb9a-b74e-49e2-97a0-0387d35ecd90`::finished: None >MainThread::DEBUG::2014-10-15 06:42:30,195::task::595::Storage.TaskManager.Task::(_updateState) Task=`a565eb9a-b74e-49e2-97a0-0387d35ecd90`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-15 06:42:30,195::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-15 06:42:30,195::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-15 06:42:30,195::task::993::Storage.TaskManager.Task::(_decref) Task=`a565eb9a-b74e-49e2-97a0-0387d35ecd90`::ref 0 aborting False >MainThread::INFO::2014-10-15 06:42:30,195::vdsm::137::vds::(run) VDSM main thread ended. Waiting for 20 other threads... >MainThread::INFO::2014-10-15 06:42:30,195::vdsm::141::vds::(run) <Thread(libvirtEventLoop, started daemon 140361975961344)> >MainThread::INFO::2014-10-15 06:42:30,195::vdsm::141::vds::(run) <Thread(Thread-21, started daemon 140361200559872)> >MainThread::INFO::2014-10-15 06:42:30,196::vdsm::141::vds::(run) <WorkerThread(Thread-10, started daemon 140361208952576)> >MainThread::INFO::2014-10-15 06:42:30,196::vdsm::141::vds::(run) <PolicyEngine(PolicyEngine, started daemon 140360487507712)> >MainThread::INFO::2014-10-15 06:42:30,196::vdsm::141::vds::(run) <WorkerThread(Thread-2, started daemon 140361959175936)> >MainThread::INFO::2014-10-15 06:42:30,196::vdsm::141::vds::(run) <WorkerThread(Thread-1, started daemon 140361967568640)> >MainThread::INFO::2014-10-15 06:42:30,196::vdsm::141::vds::(run) <WorkerThread(Thread-9, started daemon 140361695467264)> >MainThread::INFO::2014-10-15 06:42:30,196::vdsm::141::vds::(run) <WorkerThread(Thread-8, started daemon 140361703859968)> >MainThread::INFO::2014-10-15 06:42:30,196::vdsm::141::vds::(run) <HostMonitor(HostMonitor, started daemon 140360887768832)> >MainThread::INFO::2014-10-15 06:42:30,196::vdsm::141::vds::(run) <WorkerThread(Thread-7, started daemon 140361712252672)> >MainThread::INFO::2014-10-15 06:42:30,196::vdsm::141::vds::(run) <WorkerThread(Thread-6, started daemon 140361720645376)> >MainThread::INFO::2014-10-15 06:42:30,196::vdsm::141::vds::(run) <GuestManager(GuestManager, started daemon 140360529471232)> >MainThread::INFO::2014-10-15 06:42:30,196::vdsm::141::vds::(run) <WorkerThread(Thread-3, started daemon 140361745823488)> >MainThread::INFO::2014-10-15 06:42:30,196::vdsm::141::vds::(run) <GuestMonitor(GuestMonitor-rhevh-guest-rhel7.0, started daemon 140360495900416)> >MainThread::INFO::2014-10-15 06:42:30,196::vdsm::141::vds::(run) <Thread(clientIFinit, started 140360918357760)> >MainThread::INFO::2014-10-15 06:42:30,197::vdsm::141::vds::(run) <_MainThread(MainThread, started 140362212644672)> >MainThread::INFO::2014-10-15 06:42:30,197::momIF::91::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-10-15 06:42:30,197::vdsm::141::vds::(run) <MomThread(MOM, started 140360926750464)> >MainThread::INFO::2014-10-15 06:42:30,197::vmchannels::188::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-10-15 06:42:30,197::vdsm::141::vds::(run) <Listener(VM Channels Listener, started daemon 140360909965056)> >MainThread::INFO::2014-10-15 06:42:30,197::vdsm::141::vds::(run) <WorkerThread(Thread-5, started daemon 140361729038080)> >MainThread::INFO::2014-10-15 06:42:30,197::vdsm::141::vds::(run) <WorkerThread(Thread-4, started daemon 140361737430784)> >MainThread::DEBUG::2014-10-15 06:42:30,197::sampling::412::vm.Vm::(stop) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stop statistics collection >MainThread::INFO::2014-10-15 06:42:30,197::vdsm::141::vds::(run) <VmStatsThread(Thread-17, started daemon 140361192167168)> >Thread-17::DEBUG::2014-10-15 06:42:30,198::sampling::443::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stats thread finished >VM Channels Listener::INFO::2014-10-15 06:42:30,418::vmchannels::183::vds::(run) VM channels listener thread has ended. >MainThread::INFO::2014-10-15 06:42:43,336::vdsm::131::vds::(run) (PID: 14930) I am the actual vdsm 4.16.6-1.el7 hp-z220-04.qe.lab.eng.nay.redhat.com (3.10.0-123.8.1.el7.x86_64) >MainThread::DEBUG::2014-10-15 06:42:43,336::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2014-10-15 06:42:43,336::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 10, waitTimeout: 3, maxTasks: 500 >MainThread::DEBUG::2014-10-15 06:42:43,338::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt >MainThread::WARNING::2014-10-15 06:42:43,338::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-10-15 06:42:43,356::supervdsm::77::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2014-10-15 06:42:43,360::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-10-15 06:42:43,360::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) /usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type (cwd None) >MainThread::DEBUG::2014-10-15 06:42:43,381::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-10-15 06:42:43,382::hsm::442::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-10-15 06:42:43,392::hsm::474::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-10-15 06:42:43,392::hsm::475::Storage.HSM::(__cleanStorageRepository) Mount list: ['/rhev/data-center', '/rhev/data-center/mnt/10.66.128.104:_home_NFS1'] >MainThread::DEBUG::2014-10-15 06:42:43,392::hsm::477::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-10-15 06:42:43,393::hsm::520::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2014-10-15 06:42:43,394::dispatcher::46::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-10-15 06:42:43,394::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >storageRefresh::DEBUG::2014-10-15 06:42:43,412::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-15 06:42:43,429::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >storageRefresh::DEBUG::2014-10-15 06:42:43,430::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-15 06:42:43,430::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >storageRefresh::DEBUG::2014-10-15 06:42:43,435::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >MainThread::DEBUG::2014-10-15 06:42:43,435::task::595::Storage.TaskManager.Task::(_updateState) Task=`fd1d3d38-7c98-4825-842d-a8cf5c17d7fd`::moving from state init -> state preparing >MainThread::INFO::2014-10-15 06:42:43,438::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF object at 0x247c750>>) >MainThread::INFO::2014-10-15 06:42:43,438::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::DEBUG::2014-10-15 06:42:43,438::task::1191::Storage.TaskManager.Task::(prepare) Task=`fd1d3d38-7c98-4825-842d-a8cf5c17d7fd`::finished: None >MainThread::DEBUG::2014-10-15 06:42:43,438::task::595::Storage.TaskManager.Task::(_updateState) Task=`fd1d3d38-7c98-4825-842d-a8cf5c17d7fd`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-15 06:42:43,439::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-15 06:42:43,439::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-15 06:42:43,439::task::993::Storage.TaskManager.Task::(_decref) Task=`fd1d3d38-7c98-4825-842d-a8cf5c17d7fd`::ref 0 aborting False >MainThread::INFO::2014-10-15 06:42:43,439::momIF::49::MOM::(__init__) Starting up MOM >MainThread::INFO::2014-10-15 06:42:43,440::vmchannels::192::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2014-10-15 06:42:43,442::vmchannels::175::vds::(run) Starting VM channels listener thread. >MainThread::DEBUG::2014-10-15 06:42:43,453::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <rpc.BindingXMLRPC.XmlDetector instance at 0x299efc8> >clientIFinit::DEBUG::2014-10-15 06:42:43,453::libvirtconnection::150::root::(get) trying to connect libvirt >storageRefresh::DEBUG::2014-10-15 06:42:43,453::misc::751::Storage.SamplingMethod::(__call__) Returning last result >storageRefresh::DEBUG::2014-10-15 06:42:43,494::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >MainThread::DEBUG::2014-10-15 06:42:43,516::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <yajsonrpc.stompReactor.StompDetector instance at 0x7fd184180f38> >JsonRpcServer::DEBUG::2014-10-15 06:42:43,517::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Detector thread::DEBUG::2014-10-15 06:42:43,519::protocoldetector::70::vds.MultiProtocolAcceptor::(serve_forever) Acceptor running >Detector thread::DEBUG::2014-10-15 06:42:43,519::protocoldetector::72::vds.MultiProtocolAcceptor::(serve_forever) Using required_size=7 >storageRefresh::DEBUG::2014-10-15 06:42:43,544::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >clientIFinit::DEBUG::2014-10-15 06:42:43,546::clientIF::494::vds::(_recoverVm) Trying to recover c255610f-42c2-426d-9270-e4497f4a0f37 >storageRefresh::DEBUG::2014-10-15 06:42:43,546::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >clientIFinit::INFO::2014-10-15 06:42:43,547::clientIF::390::vds::(createVm) vmContainerLock acquired by vm c255610f-42c2-426d-9270-e4497f4a0f37 >storageRefresh::DEBUG::2014-10-15 06:42:43,547::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-15 06:42:43,548::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-15 06:42:43,549::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-15 06:42:43,549::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-15 06:42:43,552::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-15 06:42:43,554::misc::751::Storage.SamplingMethod::(__call__) Returning last result >storageRefresh::DEBUG::2014-10-15 06:42:43,556::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >clientIFinit::DEBUG::2014-10-15 06:42:43,556::libvirtconnection::150::root::(get) trying to connect libvirt >storageRefresh::DEBUG::2014-10-15 06:42:43,559::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size (cwd None) >clientIFinit::DEBUG::2014-10-15 06:42:43,584::clientIF::403::vds::(createVm) Total desktops after creation of c255610f-42c2-426d-9270-e4497f4a0f37 is 1 >Thread-13::DEBUG::2014-10-15 06:42:43,585::vm::2259::vm.Vm::(_startUnderlyingVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Start >Thread-13::DEBUG::2014-10-15 06:42:43,586::vm::2263::vm.Vm::(_startUnderlyingVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::_ongoingCreations acquired >Thread-13::INFO::2014-10-15 06:42:43,587::vm::3299::vm.Vm::(_run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::VM wrapper has started >Thread-13::WARNING::2014-10-15 06:42:43,587::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': u'unix', 'alias': u'channel0', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'1'}}' found >Thread-13::WARNING::2014-10-15 06:42:43,589::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': u'unix', 'alias': u'channel1', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'2'}}' found >Thread-13::WARNING::2014-10-15 06:42:43,589::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': u'spicevmc', 'alias': u'channel2', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'3'}}' found >Thread-13::DEBUG::2014-10-15 06:42:43,591::task::595::Storage.TaskManager.Task::(_updateState) Task=`995f78f9-2d60-4283-979d-4225aa81f30f`::moving from state init -> state preparing >Thread-13::INFO::2014-10-15 06:42:43,591::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-13::ERROR::2014-10-15 06:42:43,592::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-13::ERROR::2014-10-15 06:42:43,592::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-13::DEBUG::2014-10-15 06:42:43,593::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >Thread-13::DEBUG::2014-10-15 06:42:43,594::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 (cwd None) >storageRefresh::DEBUG::2014-10-15 06:42:43,614::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-15 06:42:43,615::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >storageRefresh::DEBUG::2014-10-15 06:42:43,616::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-13::DEBUG::2014-10-15 06:42:43,628::lvm::288::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume group "55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4" not found\n Skipping volume group 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4\n'; <rc> = 5 >Thread-13::WARNING::2014-10-15 06:42:43,631::lvm::370::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4" not found', ' Skipping volume group 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4'] >Thread-15::DEBUG::2014-10-15 06:42:43,635::__init__::159::IOProcessClient::(_run) Starting IOProcess... >Thread-16::DEBUG::2014-10-15 06:42:43,640::__init__::232::IOProcess::(_processLogs) Closing unrelated FDs... >Thread-16::DEBUG::2014-10-15 06:42:43,640::__init__::232::IOProcess::(_processLogs) Opening communication channels... >Thread-16::DEBUG::2014-10-15 06:42:43,641::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:42:43,641::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:42:43,641::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:42:43,641::__init__::232::IOProcess::(_processLogs) (1) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:42:43,642::__init__::232::IOProcess::(_processLogs) (1) Queuing response >storageRefresh::DEBUG::2014-10-15 06:42:43,643::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-15 06:42:43,645::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-15 06:42:43,647::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags (cwd None) >Thread-13::DEBUG::2014-10-15 06:42:43,679::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 06:42:43,680::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:42:43,681::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:42:43,681::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:42:43,682::__init__::232::IOProcess::(_processLogs) (2) Got request for method 'touch' >storageRefresh::DEBUG::2014-10-15 06:42:43,690::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-15 06:42:43,691::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Data >storageRefresh::DEBUG::2014-10-15 06:42:43,691::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Logging >storageRefresh::DEBUG::2014-10-15 06:42:43,692::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Config >storageRefresh::DEBUG::2014-10-15 06:42:43,692::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Swap >storageRefresh::DEBUG::2014-10-15 06:42:43,693::hsm::387::Storage.HSM::(storageRefresh) HSM is ready >Thread-16::DEBUG::2014-10-15 06:42:43,710::__init__::232::IOProcess::(_processLogs) (2) Queuing response >Thread-13::DEBUG::2014-10-15 06:42:43,711::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-16::DEBUG::2014-10-15 06:42:43,712::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:42:43,712::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:42:43,712::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:42:43,713::__init__::232::IOProcess::(_processLogs) (3) Got request for method 'readfile' >Thread-16::DEBUG::2014-10-15 06:42:43,714::__init__::232::IOProcess::(_processLogs) (3) Queuing response >Thread-13::DEBUG::2014-10-15 06:42:43,715::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=domain1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-0000000003e1', 'REMOTE_PATH=10.66.128.104:/home/NFS1', 'ROLE=Master', 'SDUUID=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b90f33781d11f6e6b0b3945de3167c0719a458c1'] >Thread-16::DEBUG::2014-10-15 06:42:43,716::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:42:43,717::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:42:43,717::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:42:43,718::__init__::232::IOProcess::(_processLogs) (4) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:42:43,718::__init__::232::IOProcess::(_processLogs) (4) Queuing response >Thread-16::DEBUG::2014-10-15 06:42:43,719::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:42:43,719::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:42:43,720::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:42:43,720::__init__::232::IOProcess::(_processLogs) (5) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:42:43,721::__init__::232::IOProcess::(_processLogs) (5) Queuing response >Thread-13::DEBUG::2014-10-15 06:42:43,721::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-13::DEBUG::2014-10-15 06:42:43,722::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_imageNS' >Thread-13::DEBUG::2014-10-15 06:42:43,723::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_volumeNS' >Thread-16::DEBUG::2014-10-15 06:42:43,724::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:42:43,725::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:42:43,725::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:42:43,726::__init__::232::IOProcess::(_processLogs) (6) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:42:43,726::__init__::232::IOProcess::(_processLogs) (6) Queuing response >Thread-16::DEBUG::2014-10-15 06:42:43,728::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-13::INFO::2014-10-15 06:42:43,728::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-16::DEBUG::2014-10-15 06:42:43,728::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-13::DEBUG::2014-10-15 06:42:43,729::task::1191::Storage.TaskManager.Task::(prepare) Task=`995f78f9-2d60-4283-979d-4225aa81f30f`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-16::DEBUG::2014-10-15 06:42:43,729::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-13::DEBUG::2014-10-15 06:42:43,730::task::595::Storage.TaskManager.Task::(_updateState) Task=`995f78f9-2d60-4283-979d-4225aa81f30f`::moving from state preparing -> state finished >Thread-16::DEBUG::2014-10-15 06:42:43,730::__init__::232::IOProcess::(_processLogs) (7) Got request for method 'stat' >Thread-13::DEBUG::2014-10-15 06:42:43,730::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-10-15 06:42:43,731::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-16::DEBUG::2014-10-15 06:42:43,731::__init__::232::IOProcess::(_processLogs) (7) Queuing response >Thread-13::DEBUG::2014-10-15 06:42:43,732::task::993::Storage.TaskManager.Task::(_decref) Task=`995f78f9-2d60-4283-979d-4225aa81f30f`::ref 0 aborting False >Thread-13::DEBUG::2014-10-15 06:42:43,737::vm::1067::vm.Vm::(__init__) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Ignoring param (target, 1048576) in BalloonDevice >Thread-13::DEBUG::2014-10-15 06:42:43,833::vm::5301::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Looking for drive with attributes {'name': u'hdc', 'bootOrder': '', 'boot': [], 'readonly': True, 'address': {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, 'path': '', 'type': u'cdrom'} >Thread-13::DEBUG::2014-10-15 06:42:43,834::vm::5322::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'hdc', u'hdc'), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, True), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}), 'path': ('', ''), 'type': (u'cdrom', u'cdrom')} >Thread-13::DEBUG::2014-10-15 06:42:43,835::vm::5341::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'hdc', None), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, None), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, None), 'path': ('', None), 'type': (u'cdrom', None)} >Thread-13::DEBUG::2014-10-15 06:42:43,837::vm::5301::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Looking for drive with attributes {'name': u'vda', 'bootOrder': u'2', 'boot': [<DOM Element: boot at 0x7fd180078710>], 'readonly': False, 'address': {u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, 'path': u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'type': u'disk'} >Thread-13::DEBUG::2014-10-15 06:42:43,838::vm::5322::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'vda', u'vda'), 'bootOrder': (u'2', u'2'), 'boot': ([<DOM Element: boot at 0x7fd180078710>], None), 'readonly': (False, False), 'address': ({u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, {u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}), 'path': (u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb'), 'type': (u'disk', u'disk')} >Thread-13::DEBUG::2014-10-15 06:42:43,838::vm::5341::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'vda', None), 'bootOrder': (u'2', None), 'boot': ([<DOM Element: boot at 0x7fd180078710>], None), 'readonly': (False, None), 'address': ({u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, None), 'path': (u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', None), 'type': (u'disk', None)} >Thread-13::DEBUG::2014-10-15 06:42:44,032::sampling::405::vm.Vm::(start) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Start statistics collection >Thread-17::DEBUG::2014-10-15 06:42:44,033::sampling::434::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stats thread started >Thread-13::DEBUG::2014-10-15 06:42:44,035::vmchannels::199::vds::(register) Add fileno 9 to listener's channels. >Thread-13::WARNING::2014-10-15 06:42:44,037::vm::3971::vm.Vm::(_readPauseCode) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::_readPauseCode unsupported by libvirt vm >Thread-13::DEBUG::2014-10-15 06:42:44,040::vm::2289::vm.Vm::(_startUnderlyingVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::_ongoingCreations released >Thread-17::DEBUG::2014-10-15 06:42:44,042::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 74 edom: 10 level: 2 message: argument unsupported: QEMU driver does not support <metadata> element >Thread-17::ERROR::2014-10-15 06:42:44,043::vm::373::vm.Vm::(_sampleCpuTune) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::libvirt does not support metadata >VM Channels Listener::DEBUG::2014-10-15 06:42:44,444::vmchannels::107::vds::(_do_add_channels) fileno 9 was added to unconnected channels. >VM Channels Listener::DEBUG::2014-10-15 06:42:44,444::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 9. >VM Channels Listener::DEBUG::2014-10-15 06:42:44,445::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:42:44,445::guestagent::203::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Connected to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:42:44,445::vmchannels::149::vds::(_handle_unconnected) Connecting to fileno 9 succeeded. >clientIFinit::DEBUG::2014-10-15 06:42:44,587::task::595::Storage.TaskManager.Task::(_updateState) Task=`cdf4a1fa-f5c7-4177-b9d5-280992957657`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:42:44,587::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:42:44,587::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:44,587::task::1191::Storage.TaskManager.Task::(prepare) Task=`cdf4a1fa-f5c7-4177-b9d5-280992957657`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:44,587::task::595::Storage.TaskManager.Task::(_updateState) Task=`cdf4a1fa-f5c7-4177-b9d5-280992957657`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:42:44,587::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:42:44,587::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:42:44,588::task::993::Storage.TaskManager.Task::(_decref) Task=`cdf4a1fa-f5c7-4177-b9d5-280992957657`::ref 0 aborting False >clientIFinit::DEBUG::2014-10-15 06:42:49,591::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6f23503-0db0-4efb-92ba-5fdce2fc5148`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:42:49,592::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:42:49,592::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:49,592::task::1191::Storage.TaskManager.Task::(prepare) Task=`b6f23503-0db0-4efb-92ba-5fdce2fc5148`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:49,592::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6f23503-0db0-4efb-92ba-5fdce2fc5148`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:42:49,592::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:42:49,593::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:42:49,593::task::993::Storage.TaskManager.Task::(_decref) Task=`b6f23503-0db0-4efb-92ba-5fdce2fc5148`::ref 0 aborting False >clientIFinit::DEBUG::2014-10-15 06:42:54,593::task::595::Storage.TaskManager.Task::(_updateState) Task=`14fa6b7d-58f2-45aa-b969-bef0eb1b6f18`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:42:54,594::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:42:54,594::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:54,594::task::1191::Storage.TaskManager.Task::(prepare) Task=`14fa6b7d-58f2-45aa-b969-bef0eb1b6f18`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:54,594::task::595::Storage.TaskManager.Task::(_updateState) Task=`14fa6b7d-58f2-45aa-b969-bef0eb1b6f18`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:42:54,594::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:42:54,594::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:42:54,595::task::993::Storage.TaskManager.Task::(_decref) Task=`14fa6b7d-58f2-45aa-b969-bef0eb1b6f18`::ref 0 aborting False >clientIFinit::DEBUG::2014-10-15 06:42:59,599::task::595::Storage.TaskManager.Task::(_updateState) Task=`57e46f0b-eaaf-4ae6-8d4c-a5ae5f9c4b93`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:42:59,600::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:42:59,600::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:59,600::task::1191::Storage.TaskManager.Task::(prepare) Task=`57e46f0b-eaaf-4ae6-8d4c-a5ae5f9c4b93`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:42:59,600::task::595::Storage.TaskManager.Task::(_updateState) Task=`57e46f0b-eaaf-4ae6-8d4c-a5ae5f9c4b93`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:42:59,600::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:42:59,600::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:42:59,601::task::993::Storage.TaskManager.Task::(_decref) Task=`57e46f0b-eaaf-4ae6-8d4c-a5ae5f9c4b93`::ref 0 aborting False >clientIFinit::DEBUG::2014-10-15 06:43:04,606::task::595::Storage.TaskManager.Task::(_updateState) Task=`7eaed26f-2bcd-4c77-a8c0-5b375c7f6b82`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:43:04,606::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:43:04,607::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:04,607::task::1191::Storage.TaskManager.Task::(prepare) Task=`7eaed26f-2bcd-4c77-a8c0-5b375c7f6b82`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:04,607::task::595::Storage.TaskManager.Task::(_updateState) Task=`7eaed26f-2bcd-4c77-a8c0-5b375c7f6b82`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:43:04,607::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:43:04,607::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:43:04,608::task::993::Storage.TaskManager.Task::(_decref) Task=`7eaed26f-2bcd-4c77-a8c0-5b375c7f6b82`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:08,545::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:08,546::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:08,546::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 06:43:09,613::task::595::Storage.TaskManager.Task::(_updateState) Task=`d441a87b-8ca2-40f4-893e-db623a367e43`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:43:09,613::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:43:09,613::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:09,614::task::1191::Storage.TaskManager.Task::(prepare) Task=`d441a87b-8ca2-40f4-893e-db623a367e43`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:09,614::task::595::Storage.TaskManager.Task::(_updateState) Task=`d441a87b-8ca2-40f4-893e-db623a367e43`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:43:09,614::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:43:09,614::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:43:09,614::task::993::Storage.TaskManager.Task::(_decref) Task=`d441a87b-8ca2-40f4-893e-db623a367e43`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:13,552::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:13,552::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:13,553::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:43:14,476::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 9. >clientIFinit::DEBUG::2014-10-15 06:43:14,620::task::595::Storage.TaskManager.Task::(_updateState) Task=`66f3a56b-5adf-4035-8a99-bb5d74a7d251`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:43:14,620::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:43:14,620::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:14,621::task::1191::Storage.TaskManager.Task::(prepare) Task=`66f3a56b-5adf-4035-8a99-bb5d74a7d251`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:14,621::task::595::Storage.TaskManager.Task::(_updateState) Task=`66f3a56b-5adf-4035-8a99-bb5d74a7d251`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:43:14,621::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:43:14,621::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:43:14,621::task::993::Storage.TaskManager.Task::(_decref) Task=`66f3a56b-5adf-4035-8a99-bb5d74a7d251`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:18,558::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:18,559::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:18,559::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 06:43:19,627::task::595::Storage.TaskManager.Task::(_updateState) Task=`943883a2-e681-42e8-ad9c-e8e93a6e51d9`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:43:19,627::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:43:19,627::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:19,627::task::1191::Storage.TaskManager.Task::(prepare) Task=`943883a2-e681-42e8-ad9c-e8e93a6e51d9`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:19,627::task::595::Storage.TaskManager.Task::(_updateState) Task=`943883a2-e681-42e8-ad9c-e8e93a6e51d9`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:43:19,628::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:43:19,628::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:43:19,628::task::993::Storage.TaskManager.Task::(_decref) Task=`943883a2-e681-42e8-ad9c-e8e93a6e51d9`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:23,565::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:23,565::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:23,566::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 06:43:24,633::task::595::Storage.TaskManager.Task::(_updateState) Task=`46e40277-4807-44cf-9fdd-88632d9d6241`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:43:24,633::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:43:24,634::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:24,634::task::1191::Storage.TaskManager.Task::(prepare) Task=`46e40277-4807-44cf-9fdd-88632d9d6241`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:24,634::task::595::Storage.TaskManager.Task::(_updateState) Task=`46e40277-4807-44cf-9fdd-88632d9d6241`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:43:24,634::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:43:24,634::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:43:24,634::task::993::Storage.TaskManager.Task::(_decref) Task=`46e40277-4807-44cf-9fdd-88632d9d6241`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:28,570::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:28,571::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:28,571::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 06:43:29,640::task::595::Storage.TaskManager.Task::(_updateState) Task=`f7c1566b-5a0e-47c0-8ef9-7dd2ae344e71`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:43:29,640::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:43:29,640::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:29,641::task::1191::Storage.TaskManager.Task::(prepare) Task=`f7c1566b-5a0e-47c0-8ef9-7dd2ae344e71`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:29,641::task::595::Storage.TaskManager.Task::(_updateState) Task=`f7c1566b-5a0e-47c0-8ef9-7dd2ae344e71`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:43:29,641::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:43:29,641::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:43:29,641::task::993::Storage.TaskManager.Task::(_decref) Task=`f7c1566b-5a0e-47c0-8ef9-7dd2ae344e71`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:33,574::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:33,574::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:33,575::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 06:43:34,646::task::595::Storage.TaskManager.Task::(_updateState) Task=`2acad99f-5f9f-4aa0-b26f-27c50e1b2874`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:43:34,647::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:43:34,647::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:34,647::task::1191::Storage.TaskManager.Task::(prepare) Task=`2acad99f-5f9f-4aa0-b26f-27c50e1b2874`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:34,647::task::595::Storage.TaskManager.Task::(_updateState) Task=`2acad99f-5f9f-4aa0-b26f-27c50e1b2874`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:43:34,647::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:43:34,647::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:43:34,648::task::993::Storage.TaskManager.Task::(_decref) Task=`2acad99f-5f9f-4aa0-b26f-27c50e1b2874`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:38,581::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:38,581::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:38,582::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 06:43:39,653::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2525204-b74a-4961-8926-8d6bf67a9086`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:43:39,653::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:43:39,654::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:39,654::task::1191::Storage.TaskManager.Task::(prepare) Task=`b2525204-b74a-4961-8926-8d6bf67a9086`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:39,654::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2525204-b74a-4961-8926-8d6bf67a9086`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:43:39,654::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:43:39,654::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:43:39,654::task::993::Storage.TaskManager.Task::(_decref) Task=`b2525204-b74a-4961-8926-8d6bf67a9086`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:43,587::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:43,588::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:43,588::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:43:44,508::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 9. >clientIFinit::DEBUG::2014-10-15 06:43:44,658::task::595::Storage.TaskManager.Task::(_updateState) Task=`36c3dd03-602b-471a-a8b3-42feef158e5e`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:43:44,658::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:43:44,659::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:44,659::task::1191::Storage.TaskManager.Task::(prepare) Task=`36c3dd03-602b-471a-a8b3-42feef158e5e`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:44,659::task::595::Storage.TaskManager.Task::(_updateState) Task=`36c3dd03-602b-471a-a8b3-42feef158e5e`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:43:44,659::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:43:44,659::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:43:44,659::task::993::Storage.TaskManager.Task::(_decref) Task=`36c3dd03-602b-471a-a8b3-42feef158e5e`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:48,591::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:48,592::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:43:48,592::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 06:43:49,665::task::595::Storage.TaskManager.Task::(_updateState) Task=`a17e0a2a-ce61-4f58-aa85-f0e8339b0e76`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:43:49,665::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:43:49,665::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:49,665::task::1191::Storage.TaskManager.Task::(prepare) Task=`a17e0a2a-ce61-4f58-aa85-f0e8339b0e76`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:43:49,665::task::595::Storage.TaskManager.Task::(_updateState) Task=`a17e0a2a-ce61-4f58-aa85-f0e8339b0e76`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:43:49,666::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:43:49,666::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:43:49,666::task::993::Storage.TaskManager.Task::(_decref) Task=`a17e0a2a-ce61-4f58-aa85-f0e8339b0e76`::ref 0 aborting False >MainThread::DEBUG::2014-10-15 06:43:49,936::vdsm::58::vds::(sigtermHandler) Received signal 15 >MainThread::DEBUG::2014-10-15 06:43:49,936::protocoldetector::135::vds.MultiProtocolAcceptor::(stop) Stopping Acceptor >MainThread::INFO::2014-10-15 06:43:49,937::__init__::563::jsonrpc.JsonRpcServer::(stop) Stopping JsonRPC Server >Detector thread::DEBUG::2014-10-15 06:43:49,937::protocoldetector::106::vds.MultiProtocolAcceptor::(_cleanup) Cleaning Acceptor >Thread-16::ERROR::2014-10-15 06:43:49,937::__init__::317::IOProcessClient::(_communicate) IOProcess failure >Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 272, in _communicate >Exception: FD closed >Thread-16::DEBUG::2014-10-15 06:43:49,938::__init__::159::IOProcessClient::(_run) Starting IOProcess... >MainThread::INFO::2014-10-15 06:43:49,938::vmchannels::188::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-10-15 06:43:49,942::momIF::91::MOM::(stop) Shutting down MOM >MainThread::DEBUG::2014-10-15 06:43:49,943::task::595::Storage.TaskManager.Task::(_updateState) Task=`48af7c82-bd92-4c61-8dd0-1287b9a850db`::moving from state init -> state preparing >MainThread::INFO::2014-10-15 06:43:49,943::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2014-10-15 06:43:49,943::storageServer::706::Storage.ConnectionMonitor::(_monitorConnections) Monitoring stopped >Thread-19::DEBUG::2014-10-15 06:43:49,944::__init__::232::IOProcess::(_processLogs) Opening communication channels... >MainThread::INFO::2014-10-15 06:43:49,956::domainMonitor::137::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::DEBUG::2014-10-15 06:43:49,956::taskManager::90::Storage.TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2014-10-15 06:43:49,957::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2014-10-15 06:43:49,957::task::1191::Storage.TaskManager.Task::(prepare) Task=`48af7c82-bd92-4c61-8dd0-1287b9a850db`::finished: None >MainThread::DEBUG::2014-10-15 06:43:49,957::task::595::Storage.TaskManager.Task::(_updateState) Task=`48af7c82-bd92-4c61-8dd0-1287b9a850db`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-15 06:43:49,957::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-15 06:43:49,957::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-15 06:43:49,957::task::993::Storage.TaskManager.Task::(_decref) Task=`48af7c82-bd92-4c61-8dd0-1287b9a850db`::ref 0 aborting False >MainThread::INFO::2014-10-15 06:43:49,957::vdsm::137::vds::(run) VDSM main thread ended. Waiting for 20 other threads... >MainThread::INFO::2014-10-15 06:43:49,957::vdsm::141::vds::(run) <Thread(libvirtEventLoop, started daemon 140539052263168)> >MainThread::INFO::2014-10-15 06:43:49,957::vdsm::141::vds::(run) <WorkerThread(Thread-10, started daemon 140538493785856)> >MainThread::INFO::2014-10-15 06:43:49,957::vdsm::141::vds::(run) <GuestMonitor(GuestMonitor-rhevh-guest-rhel7.0, started daemon 140537302611712)> >MainThread::INFO::2014-10-15 06:43:49,958::vmchannels::188::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-10-15 06:43:49,958::vdsm::141::vds::(run) <Listener(VM Channels Listener, started daemon 140537987811072)> >MainThread::INFO::2014-10-15 06:43:49,958::vdsm::141::vds::(run) <Thread(clientIFinit, started 140537996203776)> >MainThread::INFO::2014-10-15 06:43:49,958::vdsm::141::vds::(run) <WorkerThread(Thread-2, started daemon 140539035477760)> >MainThread::INFO::2014-10-15 06:43:49,958::vdsm::141::vds::(run) <WorkerThread(Thread-6, started daemon 140539001906944)> >MainThread::DEBUG::2014-10-15 06:43:49,958::sampling::412::vm.Vm::(stop) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stop statistics collection >Thread-17::DEBUG::2014-10-15 06:43:49,958::sampling::443::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stats thread finished >MainThread::INFO::2014-10-15 06:43:49,958::vdsm::141::vds::(run) <VmStatsThread(Thread-17, started daemon 140538477000448)> >MainThread::INFO::2014-10-15 06:43:49,958::vdsm::141::vds::(run) <Thread(Thread-19, started daemon 140538485393152)> >MainThread::INFO::2014-10-15 06:43:49,959::vdsm::141::vds::(run) <WorkerThread(Thread-9, started daemon 140538502178560)> >MainThread::INFO::2014-10-15 06:43:49,959::vdsm::141::vds::(run) <WorkerThread(Thread-8, started daemon 140538510571264)> >MainThread::INFO::2014-10-15 06:43:49,959::vdsm::141::vds::(run) <WorkerThread(Thread-4, started daemon 140539018692352)> >MainThread::INFO::2014-10-15 06:43:49,959::vdsm::141::vds::(run) <WorkerThread(Thread-5, started daemon 140539010299648)> >MainThread::INFO::2014-10-15 06:43:49,959::vdsm::141::vds::(run) <_MainThread(MainThread, started 140539288946496)> >MainThread::INFO::2014-10-15 06:43:49,959::vdsm::141::vds::(run) <WorkerThread(Thread-3, started daemon 140539027085056)> >MainThread::INFO::2014-10-15 06:43:49,959::vdsm::141::vds::(run) <GuestManager(GuestManager, started daemon 140537737668352)> >MainThread::INFO::2014-10-15 06:43:49,959::vdsm::141::vds::(run) <HostMonitor(HostMonitor, started daemon 140537762846464)> >MainThread::INFO::2014-10-15 06:43:49,959::vdsm::141::vds::(run) <PolicyEngine(PolicyEngine, started daemon 140537294219008)> >MainThread::INFO::2014-10-15 06:43:49,959::vdsm::141::vds::(run) <WorkerThread(Thread-1, started daemon 140539043870464)> >MainThread::INFO::2014-10-15 06:43:49,959::vdsm::141::vds::(run) <WorkerThread(Thread-7, started daemon 140538993514240)> >MainThread::INFO::2014-10-15 06:43:49,959::momIF::91::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-10-15 06:43:49,959::vdsm::141::vds::(run) <MomThread(MOM, started 140538004596480)> >VM Channels Listener::INFO::2014-10-15 06:43:50,515::vmchannels::183::vds::(run) VM channels listener thread has ended. >MainThread::INFO::2014-10-15 06:44:02,247::vdsm::131::vds::(run) (PID: 15435) I am the actual vdsm 4.16.6-1.el7 hp-z220-04.qe.lab.eng.nay.redhat.com (3.10.0-123.8.1.el7.x86_64) >MainThread::DEBUG::2014-10-15 06:44:02,248::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2014-10-15 06:44:02,248::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 10, waitTimeout: 3, maxTasks: 500 >MainThread::DEBUG::2014-10-15 06:44:02,250::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt >MainThread::WARNING::2014-10-15 06:44:02,250::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-10-15 06:44:02,268::supervdsm::77::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2014-10-15 06:44:02,271::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-10-15 06:44:02,272::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) /usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type (cwd None) >MainThread::DEBUG::2014-10-15 06:44:02,294::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-10-15 06:44:02,294::hsm::442::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-10-15 06:44:02,304::hsm::474::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-10-15 06:44:02,304::hsm::475::Storage.HSM::(__cleanStorageRepository) Mount list: ['/rhev/data-center', '/rhev/data-center/mnt/10.66.128.104:_home_NFS1'] >MainThread::DEBUG::2014-10-15 06:44:02,304::hsm::477::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-10-15 06:44:02,305::hsm::520::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >storageRefresh::DEBUG::2014-10-15 06:44:02,305::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2014-10-15 06:44:02,307::dispatcher::46::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-10-15 06:44:02,307::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-15 06:44:02,342::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >storageRefresh::DEBUG::2014-10-15 06:44:02,342::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-15 06:44:02,343::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >storageRefresh::DEBUG::2014-10-15 06:44:02,348::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >MainThread::DEBUG::2014-10-15 06:44:02,348::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e228500-9c10-486c-9e31-b684a51f7c97`::moving from state init -> state preparing >MainThread::INFO::2014-10-15 06:44:02,348::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF object at 0x1256750>>) >MainThread::INFO::2014-10-15 06:44:02,351::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::DEBUG::2014-10-15 06:44:02,351::task::1191::Storage.TaskManager.Task::(prepare) Task=`6e228500-9c10-486c-9e31-b684a51f7c97`::finished: None >MainThread::DEBUG::2014-10-15 06:44:02,351::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e228500-9c10-486c-9e31-b684a51f7c97`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-15 06:44:02,351::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-15 06:44:02,351::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-15 06:44:02,351::task::993::Storage.TaskManager.Task::(_decref) Task=`6e228500-9c10-486c-9e31-b684a51f7c97`::ref 0 aborting False >MainThread::INFO::2014-10-15 06:44:02,352::momIF::49::MOM::(__init__) Starting up MOM >MainThread::INFO::2014-10-15 06:44:02,353::vmchannels::192::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2014-10-15 06:44:02,355::vmchannels::175::vds::(run) Starting VM channels listener thread. >clientIFinit::DEBUG::2014-10-15 06:44:02,364::libvirtconnection::150::root::(get) trying to connect libvirt >MainThread::DEBUG::2014-10-15 06:44:02,364::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <rpc.BindingXMLRPC.XmlDetector instance at 0x187da70> >storageRefresh::DEBUG::2014-10-15 06:44:02,369::misc::751::Storage.SamplingMethod::(__call__) Returning last result >storageRefresh::DEBUG::2014-10-15 06:44:02,410::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >MainThread::DEBUG::2014-10-15 06:44:02,433::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <yajsonrpc.stompReactor.StompDetector instance at 0x7f09380ba8c0> >JsonRpcServer::DEBUG::2014-10-15 06:44:02,434::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Detector thread::DEBUG::2014-10-15 06:44:02,436::protocoldetector::70::vds.MultiProtocolAcceptor::(serve_forever) Acceptor running >Detector thread::DEBUG::2014-10-15 06:44:02,436::protocoldetector::72::vds.MultiProtocolAcceptor::(serve_forever) Using required_size=7 >storageRefresh::DEBUG::2014-10-15 06:44:02,442::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >clientIFinit::DEBUG::2014-10-15 06:44:02,444::clientIF::494::vds::(_recoverVm) Trying to recover c255610f-42c2-426d-9270-e4497f4a0f37 >storageRefresh::DEBUG::2014-10-15 06:44:02,444::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >clientIFinit::INFO::2014-10-15 06:44:02,445::clientIF::390::vds::(createVm) vmContainerLock acquired by vm c255610f-42c2-426d-9270-e4497f4a0f37 >storageRefresh::DEBUG::2014-10-15 06:44:02,446::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-15 06:44:02,447::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-15 06:44:02,447::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-15 06:44:02,449::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-15 06:44:02,450::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-15 06:44:02,453::misc::751::Storage.SamplingMethod::(__call__) Returning last result >clientIFinit::DEBUG::2014-10-15 06:44:02,453::libvirtconnection::150::root::(get) trying to connect libvirt >storageRefresh::DEBUG::2014-10-15 06:44:02,455::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-15 06:44:02,458::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size (cwd None) >clientIFinit::DEBUG::2014-10-15 06:44:02,485::clientIF::403::vds::(createVm) Total desktops after creation of c255610f-42c2-426d-9270-e4497f4a0f37 is 1 >Thread-13::DEBUG::2014-10-15 06:44:02,485::vm::2259::vm.Vm::(_startUnderlyingVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Start >Thread-13::DEBUG::2014-10-15 06:44:02,486::vm::2263::vm.Vm::(_startUnderlyingVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::_ongoingCreations acquired >Thread-13::INFO::2014-10-15 06:44:02,487::vm::3299::vm.Vm::(_run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::VM wrapper has started >Thread-13::WARNING::2014-10-15 06:44:02,488::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': u'unix', 'alias': u'channel0', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'1'}}' found >Thread-13::WARNING::2014-10-15 06:44:02,489::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': u'unix', 'alias': u'channel1', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'2'}}' found >Thread-13::WARNING::2014-10-15 06:44:02,490::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': u'spicevmc', 'alias': u'channel2', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'3'}}' found >Thread-13::DEBUG::2014-10-15 06:44:02,491::task::595::Storage.TaskManager.Task::(_updateState) Task=`89c43e30-16ea-4a7a-a588-683678f0f172`::moving from state init -> state preparing >Thread-13::INFO::2014-10-15 06:44:02,492::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-13::ERROR::2014-10-15 06:44:02,492::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-13::ERROR::2014-10-15 06:44:02,493::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-13::DEBUG::2014-10-15 06:44:02,494::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >Thread-13::DEBUG::2014-10-15 06:44:02,494::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 (cwd None) >storageRefresh::DEBUG::2014-10-15 06:44:02,503::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-15 06:44:02,504::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >storageRefresh::DEBUG::2014-10-15 06:44:02,505::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-13::DEBUG::2014-10-15 06:44:02,530::lvm::288::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume group "55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4" not found\n Skipping volume group 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4\n'; <rc> = 5 >Thread-13::WARNING::2014-10-15 06:44:02,533::lvm::370::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4" not found', ' Skipping volume group 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4'] >storageRefresh::DEBUG::2014-10-15 06:44:02,535::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-15 06:44:02,536::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-15 06:44:02,538::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags (cwd None) >Thread-15::DEBUG::2014-10-15 06:44:02,544::__init__::159::IOProcessClient::(_run) Starting IOProcess... >Thread-16::DEBUG::2014-10-15 06:44:02,550::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:02,550::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:02,551::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:02,552::__init__::232::IOProcess::(_processLogs) (1) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:44:02,552::__init__::232::IOProcess::(_processLogs) (1) Queuing response >storageRefresh::DEBUG::2014-10-15 06:44:02,567::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-15 06:44:02,569::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Data >storageRefresh::DEBUG::2014-10-15 06:44:02,570::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Logging >storageRefresh::DEBUG::2014-10-15 06:44:02,570::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Config >storageRefresh::DEBUG::2014-10-15 06:44:02,571::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Swap >storageRefresh::DEBUG::2014-10-15 06:44:02,572::hsm::387::Storage.HSM::(storageRefresh) HSM is ready >Thread-13::DEBUG::2014-10-15 06:44:02,587::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 06:44:02,588::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:02,589::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:02,589::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:02,590::__init__::232::IOProcess::(_processLogs) (2) Got request for method 'touch' >Thread-16::DEBUG::2014-10-15 06:44:02,614::__init__::232::IOProcess::(_processLogs) (2) Queuing response >Thread-13::DEBUG::2014-10-15 06:44:02,615::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-16::DEBUG::2014-10-15 06:44:02,616::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:02,617::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:02,618::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:02,618::__init__::232::IOProcess::(_processLogs) (3) Got request for method 'readfile' >Thread-16::DEBUG::2014-10-15 06:44:02,619::__init__::232::IOProcess::(_processLogs) (3) Queuing response >Thread-13::DEBUG::2014-10-15 06:44:02,620::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=domain1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-0000000003e1', 'REMOTE_PATH=10.66.128.104:/home/NFS1', 'ROLE=Master', 'SDUUID=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b90f33781d11f6e6b0b3945de3167c0719a458c1'] >Thread-16::DEBUG::2014-10-15 06:44:02,622::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:02,622::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:02,623::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:02,623::__init__::232::IOProcess::(_processLogs) (4) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:02,624::__init__::232::IOProcess::(_processLogs) (4) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:02,625::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:02,626::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:02,626::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:02,627::__init__::232::IOProcess::(_processLogs) (5) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:44:02,627::__init__::232::IOProcess::(_processLogs) (5) Queuing response >Thread-13::DEBUG::2014-10-15 06:44:02,628::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-13::DEBUG::2014-10-15 06:44:02,629::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_imageNS' >Thread-13::DEBUG::2014-10-15 06:44:02,630::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_volumeNS' >Thread-16::DEBUG::2014-10-15 06:44:02,631::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:02,631::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:02,632::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:02,632::__init__::232::IOProcess::(_processLogs) (6) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:44:02,633::__init__::232::IOProcess::(_processLogs) (6) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:02,634::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:02,635::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:02,635::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:02,636::__init__::232::IOProcess::(_processLogs) (7) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:44:02,637::__init__::232::IOProcess::(_processLogs) (7) Queuing response >Thread-13::INFO::2014-10-15 06:44:02,637::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-13::DEBUG::2014-10-15 06:44:02,638::task::1191::Storage.TaskManager.Task::(prepare) Task=`89c43e30-16ea-4a7a-a588-683678f0f172`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-13::DEBUG::2014-10-15 06:44:02,638::task::595::Storage.TaskManager.Task::(_updateState) Task=`89c43e30-16ea-4a7a-a588-683678f0f172`::moving from state preparing -> state finished >Thread-13::DEBUG::2014-10-15 06:44:02,639::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-10-15 06:44:02,639::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-10-15 06:44:02,640::task::993::Storage.TaskManager.Task::(_decref) Task=`89c43e30-16ea-4a7a-a588-683678f0f172`::ref 0 aborting False >Thread-13::DEBUG::2014-10-15 06:44:02,645::vm::1067::vm.Vm::(__init__) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Ignoring param (target, 1048576) in BalloonDevice >Thread-13::DEBUG::2014-10-15 06:44:02,736::vm::5301::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Looking for drive with attributes {'name': u'hdc', 'bootOrder': '', 'boot': [], 'readonly': True, 'address': {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, 'path': '', 'type': u'cdrom'} >Thread-13::DEBUG::2014-10-15 06:44:02,737::vm::5322::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'hdc', u'hdc'), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, True), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}), 'path': ('', ''), 'type': (u'cdrom', u'cdrom')} >Thread-13::DEBUG::2014-10-15 06:44:02,738::vm::5341::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'hdc', None), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, None), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, None), 'path': ('', None), 'type': (u'cdrom', None)} >Thread-13::DEBUG::2014-10-15 06:44:02,740::vm::5301::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Looking for drive with attributes {'name': u'vda', 'bootOrder': u'2', 'boot': [<DOM Element: boot at 0x7f09341dccb0>], 'readonly': False, 'address': {u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, 'path': u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'type': u'disk'} >Thread-13::DEBUG::2014-10-15 06:44:02,741::vm::5322::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'vda', u'vda'), 'bootOrder': (u'2', u'2'), 'boot': ([<DOM Element: boot at 0x7f09341dccb0>], None), 'readonly': (False, False), 'address': ({u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, {u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}), 'path': (u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb'), 'type': (u'disk', u'disk')} >Thread-13::DEBUG::2014-10-15 06:44:02,742::vm::5341::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'vda', None), 'bootOrder': (u'2', None), 'boot': ([<DOM Element: boot at 0x7f09341dccb0>], None), 'readonly': (False, None), 'address': ({u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, None), 'path': (u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', None), 'type': (u'disk', None)} >Thread-13::DEBUG::2014-10-15 06:44:02,958::sampling::405::vm.Vm::(start) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Start statistics collection >Thread-17::DEBUG::2014-10-15 06:44:02,959::sampling::434::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stats thread started >Thread-13::DEBUG::2014-10-15 06:44:02,961::vmchannels::199::vds::(register) Add fileno 30 to listener's channels. >Thread-13::WARNING::2014-10-15 06:44:02,962::vm::3971::vm.Vm::(_readPauseCode) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::_readPauseCode unsupported by libvirt vm >Thread-13::DEBUG::2014-10-15 06:44:02,966::vm::2289::vm.Vm::(_startUnderlyingVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::_ongoingCreations released >Thread-17::DEBUG::2014-10-15 06:44:02,969::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 74 edom: 10 level: 2 message: argument unsupported: QEMU driver does not support <metadata> element >Thread-17::ERROR::2014-10-15 06:44:02,970::vm::373::vm.Vm::(_sampleCpuTune) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::libvirt does not support metadata >VM Channels Listener::DEBUG::2014-10-15 06:44:03,357::vmchannels::107::vds::(_do_add_channels) fileno 30 was added to unconnected channels. >VM Channels Listener::DEBUG::2014-10-15 06:44:03,357::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:03,358::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:44:03,358::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >clientIFinit::DEBUG::2014-10-15 06:44:03,488::task::595::Storage.TaskManager.Task::(_updateState) Task=`15caf7fa-f4ef-4d75-94d3-94ed45b3873a`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:44:03,488::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:44:03,488::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:44:03,488::task::1191::Storage.TaskManager.Task::(prepare) Task=`15caf7fa-f4ef-4d75-94d3-94ed45b3873a`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:44:03,488::task::595::Storage.TaskManager.Task::(_updateState) Task=`15caf7fa-f4ef-4d75-94d3-94ed45b3873a`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:44:03,488::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:44:03,489::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:44:03,489::task::993::Storage.TaskManager.Task::(_decref) Task=`15caf7fa-f4ef-4d75-94d3-94ed45b3873a`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:44:04,359::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:04,360::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:44:04,360::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >VM Channels Listener::DEBUG::2014-10-15 06:44:05,362::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:05,362::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:44:05,362::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >VM Channels Listener::DEBUG::2014-10-15 06:44:06,364::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:06,364::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:44:06,365::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >VM Channels Listener::DEBUG::2014-10-15 06:44:07,367::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:07,367::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:44:07,367::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >VM Channels Listener::DEBUG::2014-10-15 06:44:08,368::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >clientIFinit::DEBUG::2014-10-15 06:44:08,494::task::595::Storage.TaskManager.Task::(_updateState) Task=`54c0b05e-6ceb-40d0-b169-37e9f60557bc`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:44:08,494::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:44:08,494::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:44:08,495::task::1191::Storage.TaskManager.Task::(prepare) Task=`54c0b05e-6ceb-40d0-b169-37e9f60557bc`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:44:08,495::task::595::Storage.TaskManager.Task::(_updateState) Task=`54c0b05e-6ceb-40d0-b169-37e9f60557bc`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:44:08,495::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:44:08,495::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:44:08,495::task::993::Storage.TaskManager.Task::(_decref) Task=`54c0b05e-6ceb-40d0-b169-37e9f60557bc`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:44:09,370::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:10,371::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:11,372::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:12,374::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:13,375::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >clientIFinit::DEBUG::2014-10-15 06:44:13,500::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e47f792-0eb8-428a-8d6f-2bc431d85feb`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:44:13,501::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:44:13,501::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:44:13,501::task::1191::Storage.TaskManager.Task::(prepare) Task=`8e47f792-0eb8-428a-8d6f-2bc431d85feb`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:44:13,501::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e47f792-0eb8-428a-8d6f-2bc431d85feb`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:44:13,501::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:44:13,501::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:44:13,501::task::993::Storage.TaskManager.Task::(_decref) Task=`8e47f792-0eb8-428a-8d6f-2bc431d85feb`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:44:14,376::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:15,378::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:16,379::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:17,381::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:18,382::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >clientIFinit::DEBUG::2014-10-15 06:44:18,507::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8db9427-fec9-43da-a295-5553f82e379d`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:44:18,507::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:44:18,507::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:44:18,507::task::1191::Storage.TaskManager.Task::(prepare) Task=`b8db9427-fec9-43da-a295-5553f82e379d`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:44:18,507::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8db9427-fec9-43da-a295-5553f82e379d`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:44:18,507::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:44:18,508::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:44:18,508::task::993::Storage.TaskManager.Task::(_decref) Task=`b8db9427-fec9-43da-a295-5553f82e379d`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:44:19,384::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:20,385::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:21,387::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:22,389::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:23,390::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >clientIFinit::DEBUG::2014-10-15 06:44:23,513::task::595::Storage.TaskManager.Task::(_updateState) Task=`0bd9b109-86d9-4c39-ae9f-9fc71eec72dc`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:44:23,513::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:44:23,514::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:44:23,514::task::1191::Storage.TaskManager.Task::(prepare) Task=`0bd9b109-86d9-4c39-ae9f-9fc71eec72dc`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 06:44:23,514::task::595::Storage.TaskManager.Task::(_updateState) Task=`0bd9b109-86d9-4c39-ae9f-9fc71eec72dc`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:44:23,514::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:44:23,514::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:44:23,515::task::993::Storage.TaskManager.Task::(_decref) Task=`0bd9b109-86d9-4c39-ae9f-9fc71eec72dc`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:44:24,391::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Detector thread::DEBUG::2014-10-15 06:44:25,288::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.66.78.3:37585 >Detector thread::DEBUG::2014-10-15 06:44:25,288::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.66.78.3:37585 >Detector thread::DEBUG::2014-10-15 06:44:25,289::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol stomp from 10.66.78.3:37585 >Detector thread::DEBUG::2014-10-15 06:44:25,289::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='CONNECT'> >Detector thread::INFO::2014-10-15 06:44:25,289::stompReactor::66::Broker.StompAdapter::(_cmd_connect) Processing CONNECT request >Detector thread::DEBUG::2014-10-15 06:44:25,289::stompReactor::338::protocoldetector.StompDetector::(handleSocket) Stomp detected from ('10.66.78.3', 37585) >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:25,290::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SUBSCRIBE'> >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:25,290::stompReactor::89::Broker.StompAdapter::(_cmd_subscribe) Subscribe command ignored >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:25,330::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-18::DEBUG::2014-10-15 06:44:25,330::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getCapabilities' in bridge with {} >JsonRpcServer::DEBUG::2014-10-15 06:44:25,330::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-18::DEBUG::2014-10-15 06:44:25,351::utils::738::root::(execCmd) /sbin/ip route show to 0.0.0.0/0 table all (cwd None) >Thread-18::DEBUG::2014-10-15 06:44:25,357::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-15 06:44:25,383::utils::738::root::(execCmd) numactl --hardware (cwd None) >Thread-18::DEBUG::2014-10-15 06:44:25,388::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-15 06:44:25,389::utils::738::root::(execCmd) sysctl -n -e kernel.numa_balancing (cwd None) >VM Channels Listener::DEBUG::2014-10-15 06:44:25,393::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-18::DEBUG::2014-10-15 06:44:25,394::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-15 06:44:25,397::caps::269::root::(getLiveMergeSupport) libvirt is missing 'VIR_DOMAIN_BLOCK_COMMIT_RELATIVE': live merge disabled >Thread-18::DEBUG::2014-10-15 06:44:25,398::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getCapabilities' in bridge with {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:67bf3ab0cc3a'}], 'FC': []}, 'packages2': {'kernel': {'release': '123.8.1.el7.x86_64', 'buildtime': 1407764269.0, 'version': '3.10.0'}, 'spice-server': {'release': '5.el7', 'buildtime': 1391017388L, 'version': '0.12.4'}, 'vdsm': {'release': '1.el7', 'buildtime': 1412523943L, 'version': '4.16.6'}, 'qemu-kvm': {'release': '60.el7_0.10', 'buildtime': 1412003348L, 'version': '1.5.3'}, 'libvirt': {'release': '29.el7_0.3', 'buildtime': 1411483944L, 'version': '1.1.1'}, 'qemu-img': {'release': '60.el7_0.10', 'buildtime': 1412003348L, 'version': '1.5.3'}, 'mom': {'release': '1.el7_0', 'buildtime': 1405326698L, 'version': '0.4.1'}}, 'numaNodeDistance': {'0': [10]}, 'cpuModel': 'Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz', 'liveMerge': 'false', 'hooks': {'before_vm_migrate_destination': {'50_vhostmd': {'md5': 'f3ee6dbf6fbd01333bd3e32afec4fbba'}}, 'after_vm_destroy': {'50_vhostmd': {'md5': '92498ed80c0219749829ecce2813fc7c'}}, 'before_vm_dehibernate': {'50_vhostmd': {'md5': 'f3ee6dbf6fbd01333bd3e32afec4fbba'}}, 'before_vm_start': {'50_hostedengine': {'md5': '45dde62155b5412eafbfff5ef265acc2'}, '50_vhostmd': {'md5': 'f3ee6dbf6fbd01333bd3e32afec4fbba'}}}, 'cpuSockets': '1', 'vmTypes': ['kvm'], 'selinux': {'mode': '1'}, 'kdumpStatus': 0, 'supportedProtocols': ['2.2', '2.3'], 'networks': {'rhevm': {'iface': 'rhevm', 'addr': '10.66.100.109', 'cfg': {'IPV6_AUTOCONF': 'no', 'IPV6INIT': 'no', 'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'PEERDNS': 'no', 'DEVICE': 'rhevm', 'PEERNTP': 'yes', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'bridged': True, 'ipv6addrs': ['fe80::a2b3:ccff:fef7:e533/64'], 'gateway': '10.66.103.254', 'bootproto4': 'dhcp', 'netmask': '255.255.252.0', 'stp': 'off', 'ipv4addrs': ['10.66.100.109/22'], 'mtu': '1500', 'ipv6gateway': '::', 'ports': ['eno1', 'vnet0']}}, 'bridges': {'rhevm': {'addr': '10.66.100.109', 'cfg': {'IPV6_AUTOCONF': 'no', 'IPV6INIT': 'no', 'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'PEERDNS': 'no', 'DEVICE': 'rhevm', 'PEERNTP': 'yes', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ipv6addrs': ['fe80::a2b3:ccff:fef7:e533/64'], 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'ipv4addrs': ['10.66.100.109/22'], 'ipv6gateway': '::', 'gateway': '10.66.103.254', 'opts': {'multicast_last_member_count': '2', 'hash_elasticity': '4', 'multicast_query_response_interval': '1000', 'group_fwd_mask': '0x0', 'multicast_snooping': '1', 'multicast_startup_query_interval': '3125', 'hello_timer': '20', 'multicast_querier_interval': '25500', 'max_age': '2000', 'hash_max': '512', 'stp_state': '0', 'topology_change_detected': '0', 'priority': '32768', 'multicast_membership_interval': '26000', 'root_path_cost': '0', 'root_port': '0', 'multicast_querier': '0', 'multicast_startup_query_count': '2', 'nf_call_iptables': '0', 'topology_change': '0', 'hello_time': '200', 'root_id': '8000.a0b3ccf7e533', 'bridge_id': '8000.a0b3ccf7e533', 'topology_change_timer': '0', 'ageing_time': '30000', 'nf_call_ip6tables': '0', 'gc_timer': '15750', 'nf_call_arptables': '0', 'group_addr': '1:80:c2:0:0:0', 'multicast_last_member_interval': '100', 'multicast_query_interval': '12500', 'tcn_timer': '0', 'multicast_router': '1', 'vlan_filtering': '0', 'forward_delay': '0'}, 'ports': ['eno1', 'vnet0']}}, 'uuid': 'A63BD200-CD0E-11E1-9290-A0B3CCF7E533', 'lastClientIface': 'rhevm', 'nics': {'eno1': {'addr': '', 'cfg': {'BRIDGE': 'rhevm', 'IPV6INIT': 'no', 'NM_CONTROLLED': 'no', 'PEERDNS': 'no', 'IPV6_AUTOCONF': 'no', 'HWADDR': 'a0:b3:cc:f7:e5:33', 'DEVICE': 'eno1', 'PEERNTP': 'yes', 'ONBOOT': 'yes'}, 'ipv6addrs': ['fe80::a2b3:ccff:fef7:e533/64'], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': 'a0:b3:cc:f7:e5:33', 'speed': 1000}}, 'software_revision': '1', 'clusterLevels': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,lahf_lm,ida,arat,epb,xsaveopt,pln,pts,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,fsgsbase,smep,erms,model_Nehalem,model_Conroe,model_coreduo,model_core2duo,model_Penryn,model_Westmere,model_n270,model_SandyBridge', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:67bf3ab0cc3a', 'netConfigDirty': 'False', 'supportedENGINEs': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'autoNumaBalancing': 0, 'reservedMem': '321', 'bondings': {}, 'software_version': '4.16', 'memSize': '7732', 'cpuSpeed': '1600.250', 'numaNodes': {u'0': {'totalMemory': '7732', 'cpus': [0, 1, 2, 3]}}, 'version_name': 'Snow Man', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0'], 'cpuThreads': '4', 'emulatedMachines': [u'pc', u'q35', u'rhel6.0.0', u'rhel6.1.0', u'rhel6.2.0', u'rhel6.3.0', u'rhel6.4.0', u'rhel6.5.0', u'none'], 'rngSources': ['random'], 'operatingSystem': {'release': '20141006.0.el7ev', 'version': '7.0', 'name': 'RHEV Hypervisor'}} >Thread-18::DEBUG::2014-10-15 06:44:25,399::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:25,402::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:25,403::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-19::DEBUG::2014-10-15 06:44:25,403::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getHardwareInfo' in bridge with {} >Thread-19::DEBUG::2014-10-15 06:44:25,428::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getHardwareInfo' in bridge with {'systemProductName': 'HP Z220 CMT Workstation', 'systemUUID': 'A63BD200-CD0E-11E1-9290-A0B3CCF7E533', 'systemSerialNumber': 'CNG229WDQR', 'systemFamily': '103C_53335X G=D', 'systemManufacturer': 'Hewlett-Packard'} >Thread-19::DEBUG::2014-10-15 06:44:25,429::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:25,496::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:25,497::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-20::DEBUG::2014-10-15 06:44:25,498::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:25,499::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:25,499::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-21::DEBUG::2014-10-15 06:44:25,500::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-21::DEBUG::2014-10-15 06:44:25,501::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:25,516::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:25,516::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-22::DEBUG::2014-10-15 06:44:25,516::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connectStorageServer' in bridge with {u'connectionParams': [{u'id': u'cd8fa491-c53f-4601-8cc6-cc31a63494e7', u'connection': u'10.66.128.104:/home/NFS1', u'iqn': u'', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u''}], u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1', u'domainType': 1} >Thread-22::DEBUG::2014-10-15 06:44:25,517::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0afe0b0-17bd-4ac4-b356-eaa6104c7e2b`::moving from state init -> state preparing >Thread-22::INFO::2014-10-15 06:44:25,518::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID=u'00000002-0002-0002-0002-0000000003e1', conList=[{u'connection': u'10.66.128.104:/home/NFS1', u'iqn': u'', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'cd8fa491-c53f-4601-8cc6-cc31a63494e7', u'port': u''}], options=None) >Thread-22::DEBUG::2014-10-15 06:44:25,524::hsm::2372::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.66.128.104:_home_NFS1 >Thread-16::DEBUG::2014-10-15 06:44:25,525::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:25,525::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:25,525::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:25,525::__init__::232::IOProcess::(_processLogs) (8) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:44:25,525::__init__::232::IOProcess::(_processLogs) (8) Queuing response >Thread-22::DEBUG::2014-10-15 06:44:25,526::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: (u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4',) >Thread-22::DEBUG::2014-10-15 06:44:25,526::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4: storage.nfsSD.findDomain} >Thread-22::INFO::2014-10-15 06:44:25,526::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': u'cd8fa491-c53f-4601-8cc6-cc31a63494e7'}]} >Thread-22::DEBUG::2014-10-15 06:44:25,526::task::1191::Storage.TaskManager.Task::(prepare) Task=`e0afe0b0-17bd-4ac4-b356-eaa6104c7e2b`::finished: {'statuslist': [{'status': 0, 'id': u'cd8fa491-c53f-4601-8cc6-cc31a63494e7'}]} >Thread-22::DEBUG::2014-10-15 06:44:25,526::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0afe0b0-17bd-4ac4-b356-eaa6104c7e2b`::moving from state preparing -> state finished >Thread-22::DEBUG::2014-10-15 06:44:25,526::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-22::DEBUG::2014-10-15 06:44:25,526::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-22::DEBUG::2014-10-15 06:44:25,526::task::993::Storage.TaskManager.Task::(_decref) Task=`e0afe0b0-17bd-4ac4-b356-eaa6104c7e2b`::ref 0 aborting False >Thread-22::DEBUG::2014-10-15 06:44:25,526::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connectStorageServer' in bridge with [{'status': 0, 'id': u'cd8fa491-c53f-4601-8cc6-cc31a63494e7'}] >Thread-22::DEBUG::2014-10-15 06:44:25,527::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:25,533::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:25,534::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-23::DEBUG::2014-10-15 06:44:25,534::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connect' in bridge with {u'masterVersion': 1, u'domainDict': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': u'active'}, u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1', u'scsiKey': u'00000002-0002-0002-0002-0000000003e1', u'masterSdUUID': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', u'hostID': 2} >Thread-23::DEBUG::2014-10-15 06:44:25,535::task::595::Storage.TaskManager.Task::(_updateState) Task=`39dae4b9-5011-4acd-9a93-90138587f14c`::moving from state init -> state preparing >Thread-23::INFO::2014-10-15 06:44:25,535::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID=u'00000002-0002-0002-0002-0000000003e1', hostID=2, msdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', masterVersion=1, domainsMap={u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': u'active'}, options=None) >Thread-23::DEBUG::2014-10-15 06:44:25,535::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`0d25b801-8586-4348-9a82-e7087a5f012d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-23::DEBUG::2014-10-15 06:44:25,535::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-23::DEBUG::2014-10-15 06:44:25,536::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-23::DEBUG::2014-10-15 06:44:25,536::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`0d25b801-8586-4348-9a82-e7087a5f012d`::Granted request >Thread-23::DEBUG::2014-10-15 06:44:25,536::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`44b2735f-faee-4d81-b42a-6e90f590e80a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1059' at '_connectStoragePool' >Thread-23::DEBUG::2014-10-15 06:44:25,536::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'exclusive' >Thread-23::DEBUG::2014-10-15 06:44:25,536::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'exclusive' (1 active user) >Thread-23::DEBUG::2014-10-15 06:44:25,536::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`44b2735f-faee-4d81-b42a-6e90f590e80a`::Granted request >Thread-23::DEBUG::2014-10-15 06:44:25,537::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 1 and domains map to {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': u'active'} >Thread-23::INFO::2014-10-15 06:44:25,537::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 1 and domains map {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': u'Active'} >Thread-23::INFO::2014-10-15 06:44:25,537::sp::133::Storage.StoragePool::(setBackend) updating pool 00000002-0002-0002-0002-0000000003e1 backend from type NoneType instance 0x7f098db4df00 to type StoragePoolMemoryBackend instance 0x7f090c23d4c8 >Thread-23::INFO::2014-10-15 06:44:25,537::sp::631::Storage.StoragePool::(connect) Connect host #2 to the storage pool 00000002-0002-0002-0002-0000000003e1 with master domain: 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 (ver = 1) >Thread-23::DEBUG::2014-10-15 06:44:25,537::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-23::DEBUG::2014-10-15 06:44:25,537::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-23::DEBUG::2014-10-15 06:44:25,537::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-23::DEBUG::2014-10-15 06:44:25,537::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-23::DEBUG::2014-10-15 06:44:25,538::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-23::DEBUG::2014-10-15 06:44:25,538::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-23::DEBUG::2014-10-15 06:44:25,538::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-23::DEBUG::2014-10-15 06:44:25,538::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-23::DEBUG::2014-10-15 06:44:25,538::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-23::DEBUG::2014-10-15 06:44:25,538::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-23::DEBUG::2014-10-15 06:44:25,538::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-23::DEBUG::2014-10-15 06:44:25,538::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-23::DEBUG::2014-10-15 06:44:25,549::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-23::DEBUG::2014-10-15 06:44:25,550::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-23::DEBUG::2014-10-15 06:44:25,561::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-23::DEBUG::2014-10-15 06:44:25,561::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-23::DEBUG::2014-10-15 06:44:25,562::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-23::DEBUG::2014-10-15 06:44:25,562::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-23::DEBUG::2014-10-15 06:44:25,562::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-23::DEBUG::2014-10-15 06:44:25,562::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-23::DEBUG::2014-10-15 06:44:25,562::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-23::DEBUG::2014-10-15 06:44:25,562::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-16::DEBUG::2014-10-15 06:44:25,563::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:25,563::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:25,563::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:25,564::__init__::232::IOProcess::(_processLogs) (9) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:44:25,564::__init__::232::IOProcess::(_processLogs) (9) Queuing response >Thread-23::DEBUG::2014-10-15 06:44:25,570::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 06:44:25,571::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:25,571::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:25,571::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:25,571::__init__::232::IOProcess::(_processLogs) (10) Got request for method 'touch' >Thread-16::DEBUG::2014-10-15 06:44:25,616::__init__::232::IOProcess::(_processLogs) (10) Queuing response >Thread-23::DEBUG::2014-10-15 06:44:25,617::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-16::DEBUG::2014-10-15 06:44:25,617::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:25,617::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:25,617::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:25,618::__init__::232::IOProcess::(_processLogs) (11) Got request for method 'readfile' >Thread-16::DEBUG::2014-10-15 06:44:25,618::__init__::232::IOProcess::(_processLogs) (11) Queuing response >Thread-23::DEBUG::2014-10-15 06:44:25,618::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=domain1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-0000000003e1', 'REMOTE_PATH=10.66.128.104:/home/NFS1', 'ROLE=Master', 'SDUUID=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b90f33781d11f6e6b0b3945de3167c0719a458c1'] >Thread-16::DEBUG::2014-10-15 06:44:25,619::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:25,619::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:25,619::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:25,619::__init__::232::IOProcess::(_processLogs) (12) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:25,619::__init__::232::IOProcess::(_processLogs) (12) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:25,619::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:25,619::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:25,619::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:25,620::__init__::232::IOProcess::(_processLogs) (13) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:44:25,620::__init__::232::IOProcess::(_processLogs) (13) Queuing response >Thread-23::DEBUG::2014-10-15 06:44:25,620::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-23::INFO::2014-10-15 06:44:25,620::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_imageNS already registered >Thread-23::INFO::2014-10-15 06:44:25,620::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_volumeNS already registered >Thread-23::DEBUG::2014-10-15 06:44:25,620::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 verified, version 1 >Thread-23::DEBUG::2014-10-15 06:44:25,620::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-23::DEBUG::2014-10-15 06:44:25,620::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-23::INFO::2014-10-15 06:44:25,620::domainMonitor::100::Storage.DomainMonitor::(startMonitoring) Start monitoring 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-23::DEBUG::2014-10-15 06:44:25,621::sp::1413::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `00000002-0002-0002-0002-0000000003e1` started monitoring domain `55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4` >Thread-25::DEBUG::2014-10-15 06:44:25,621::domainMonitor::182::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-23::DEBUG::2014-10-15 06:44:25,621::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-23::DEBUG::2014-10-15 06:44:25,622::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/00000002-0002-0002-0002-0000000003e1 >Thread-16::DEBUG::2014-10-15 06:44:25,622::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:25,623::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:25,623::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:25,624::__init__::232::IOProcess::(_processLogs) (14) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:44:25,624::__init__::232::IOProcess::(_processLogs) (14) Queuing response >Thread-23::DEBUG::2014-10-15 06:44:25,625::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-25::DEBUG::2014-10-15 06:44:25,626::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-23::DEBUG::2014-10-15 06:44:25,632::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-25::DEBUG::2014-10-15 06:44:25,632::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000350039 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:44:25,637::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:25,637::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:25,637::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:25,637::__init__::232::IOProcess::(_processLogs) (15) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:44:25,638::__init__::232::IOProcess::(_processLogs) (15) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:25,638::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:25,638::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:25,638::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:25,638::__init__::232::IOProcess::(_processLogs) (16) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:25,639::__init__::232::IOProcess::(_processLogs) (16) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:25,639::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:25,639::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:25,639::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:25,639::__init__::232::IOProcess::(_processLogs) (17) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:25,639::__init__::232::IOProcess::(_processLogs) (17) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:25,640::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:25,640::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:25,640::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:25,640::__init__::232::IOProcess::(_processLogs) (18) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:25,640::__init__::232::IOProcess::(_processLogs) (18) Queuing response >Thread-25::DEBUG::2014-10-15 06:44:25,641::domainMonitor::264::Storage.DomainMonitorThread::(_monitorDomain) Domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 changed its status to Valid >Thread-26::DEBUG::2014-10-15 06:44:25,641::misc::821::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-26::DEBUG::2014-10-15 06:44:25,641::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-26::DEBUG::2014-10-15 06:44:25,642::misc::841::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-27::INFO::2014-10-15 06:44:25,642::clientIF::136::vds::(contEIOVms) vmContainerLock acquired >Thread-23::DEBUG::2014-10-15 06:44:25,662::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-23::DEBUG::2014-10-15 06:44:25,662::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-16::DEBUG::2014-10-15 06:44:25,663::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:25,663::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:25,663::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:25,663::__init__::232::IOProcess::(_processLogs) (19) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:44:25,663::__init__::232::IOProcess::(_processLogs) (19) Queuing response >Thread-23::INFO::2014-10-15 06:44:25,664::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 to /rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-23::DEBUG::2014-10-15 06:44:25,664::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 to /rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-23::INFO::2014-10-15 06:44:25,664::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 to /rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd >Thread-23::DEBUG::2014-10-15 06:44:25,664::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 to /rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd >Thread-23::DEBUG::2014-10-15 06:44:25,665::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 5, waitTimeout: 3, maxTasks: 500 >Thread-23::DEBUG::2014-10-15 06:44:25,667::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /usr/bin/dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/outbox iflag=direct,fullblock bs=512 count=8 skip=16 (cwd None) >Thread-23::DEBUG::2014-10-15 06:44:25,676::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00289692 s, 1.4 MB/s\n'; <rc> = 0 >Thread-23::INFO::2014-10-15 06:44:25,676::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/usr/bin/dd', 'of=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=16'] >Thread-23::DEBUG::2014-10-15 06:44:25,677::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /usr/bin/dd of=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=16 (cwd None) >Thread-23::DEBUG::2014-10-15 06:44:25,787::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.105224 s, 38.9 kB/s\n'; <rc> = 0 >Thread-23::DEBUG::2014-10-15 06:44:25,788::storage_mailbox::207::Storage.Mailbox.HSM::(__init__) HSM_MailboxMonitor created for pool 00000002-0002-0002-0002-0000000003e1 >Thread-23::DEBUG::2014-10-15 06:44:25,788::sp::462::Storage.StoragePool::(__createMailboxMonitor) HSM mailbox ready for pool 00000002-0002-0002-0002-0000000003e1 on master domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-23::DEBUG::2014-10-15 06:44:25,788::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-23::DEBUG::2014-10-15 06:44:25,789::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-23::DEBUG::2014-10-15 06:44:25,789::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-23::DEBUG::2014-10-15 06:44:25,789::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-23::DEBUG::2014-10-15 06:44:25,789::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-23::DEBUG::2014-10-15 06:44:25,789::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-23::DEBUG::2014-10-15 06:44:25,789::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-23::DEBUG::2014-10-15 06:44:25,789::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-23::INFO::2014-10-15 06:44:25,789::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-23::DEBUG::2014-10-15 06:44:25,790::task::1191::Storage.TaskManager.Task::(prepare) Task=`39dae4b9-5011-4acd-9a93-90138587f14c`::finished: True >Thread-23::DEBUG::2014-10-15 06:44:25,790::task::595::Storage.TaskManager.Task::(_updateState) Task=`39dae4b9-5011-4acd-9a93-90138587f14c`::moving from state preparing -> state finished >Thread-23::DEBUG::2014-10-15 06:44:25,790::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-23::DEBUG::2014-10-15 06:44:25,790::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-23::DEBUG::2014-10-15 06:44:25,790::task::993::Storage.TaskManager.Task::(_decref) Task=`39dae4b9-5011-4acd-9a93-90138587f14c`::ref 0 aborting False >Thread-23::DEBUG::2014-10-15 06:44:25,790::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connect' in bridge with True >Thread-23::DEBUG::2014-10-15 06:44:25,790::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:25,792::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:25,792::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-35::DEBUG::2014-10-15 06:44:25,794::task::595::Storage.TaskManager.Task::(_updateState) Task=`8af3fdee-9da4-4530-8128-b429dcc3a344`::moving from state init -> state preparing >Thread-35::INFO::2014-10-15 06:44:25,794::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-35::INFO::2014-10-15 06:44:25,794::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000350039', 'lastCheck': '0.2', 'valid': True}} >Thread-35::DEBUG::2014-10-15 06:44:25,794::task::1191::Storage.TaskManager.Task::(prepare) Task=`8af3fdee-9da4-4530-8128-b429dcc3a344`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000350039', 'lastCheck': '0.2', 'valid': True}} >Thread-35::DEBUG::2014-10-15 06:44:25,794::task::595::Storage.TaskManager.Task::(_updateState) Task=`8af3fdee-9da4-4530-8128-b429dcc3a344`::moving from state preparing -> state finished >Thread-35::DEBUG::2014-10-15 06:44:25,794::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-35::DEBUG::2014-10-15 06:44:25,794::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-35::DEBUG::2014-10-15 06:44:25,794::task::993::Storage.TaskManager.Task::(_decref) Task=`8af3fdee-9da4-4530-8128-b429dcc3a344`::ref 0 aborting False >Thread-35::ERROR::2014-10-15 06:44:25,795::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-35::DEBUG::2014-10-15 06:44:25,796::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:25,806::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:25,806::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-36::DEBUG::2014-10-15 06:44:25,806::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.setMOMPolicyParameters' in bridge with {u'key_value_store': {u'balloonEnabled': False, u'ksmEnabled': True}} >Thread-36::DEBUG::2014-10-15 06:44:25,859::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.setMOMPolicyParameters' in bridge with True >Thread-36::DEBUG::2014-10-15 06:44:25,859::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:44:26,395::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:27,397::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:27,463::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:27,464::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:27,464::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:44:28,398::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >clientIFinit::DEBUG::2014-10-15 06:44:28,518::task::595::Storage.TaskManager.Task::(_updateState) Task=`5315a583-8b7b-408c-8da5-d19e86266965`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:44:28,518::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 06:44:28,519::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': [u'00000002-0002-0002-0002-0000000003e1']} >clientIFinit::DEBUG::2014-10-15 06:44:28,519::task::1191::Storage.TaskManager.Task::(prepare) Task=`5315a583-8b7b-408c-8da5-d19e86266965`::finished: {'poollist': [u'00000002-0002-0002-0002-0000000003e1']} >clientIFinit::DEBUG::2014-10-15 06:44:28,519::task::595::Storage.TaskManager.Task::(_updateState) Task=`5315a583-8b7b-408c-8da5-d19e86266965`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:44:28,519::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:44:28,519::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:44:28,519::task::993::Storage.TaskManager.Task::(_decref) Task=`5315a583-8b7b-408c-8da5-d19e86266965`::ref 0 aborting False >clientIFinit::WARNING::2014-10-15 06:44:28,519::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': u'unix', 'alias': u'channel0', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'1'}}' found >clientIFinit::WARNING::2014-10-15 06:44:28,520::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': u'unix', 'alias': u'channel1', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'2'}}' found >clientIFinit::WARNING::2014-10-15 06:44:28,520::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': u'spicevmc', 'alias': u'channel2', 'type': u'channel', 'address': {u'bus': u'0', u'controller': u'0', u'type': u'virtio-serial', u'port': u'3'}}' found >clientIFinit::DEBUG::2014-10-15 06:44:28,520::task::595::Storage.TaskManager.Task::(_updateState) Task=`37667f50-d39c-40b5-aa3a-5833b0aef825`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:44:28,520::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:44:28,521::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:28,521::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:28,521::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:28,521::__init__::232::IOProcess::(_processLogs) (20) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:44:28,521::__init__::232::IOProcess::(_processLogs) (20) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:28,522::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:28,522::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:28,522::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:28,522::__init__::232::IOProcess::(_processLogs) (21) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:44:28,522::__init__::232::IOProcess::(_processLogs) (21) Queuing response >clientIFinit::INFO::2014-10-15 06:44:28,522::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >clientIFinit::DEBUG::2014-10-15 06:44:28,522::task::1191::Storage.TaskManager.Task::(prepare) Task=`37667f50-d39c-40b5-aa3a-5833b0aef825`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >clientIFinit::DEBUG::2014-10-15 06:44:28,522::task::595::Storage.TaskManager.Task::(_updateState) Task=`37667f50-d39c-40b5-aa3a-5833b0aef825`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:44:28,522::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 06:44:28,523::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:44:28,523::task::993::Storage.TaskManager.Task::(_decref) Task=`37667f50-d39c-40b5-aa3a-5833b0aef825`::ref 0 aborting False >clientIFinit::INFO::2014-10-15 06:44:28,523::clientIF::331::vds::(prepareVolumePath) prepared volume path: >clientIFinit::DEBUG::2014-10-15 06:44:28,523::task::595::Storage.TaskManager.Task::(_updateState) Task=`da4cd80b-9dfa-42e8-bded-10a7c158bba3`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 06:44:28,523::logUtils::44::dispatcher::(wrapper) Run and protect: prepareImage(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', leafUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb') >clientIFinit::DEBUG::2014-10-15 06:44:28,523::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4`ReqID=`3d619ca1-a2cc-41c7-b765-f9f0895af96f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '3210' at 'prepareImage' >clientIFinit::DEBUG::2014-10-15 06:44:28,523::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' for lock type 'shared' >clientIFinit::DEBUG::2014-10-15 06:44:28,524::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' is free. Now locking as 'shared' (1 active user) >clientIFinit::DEBUG::2014-10-15 06:44:28,524::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4`ReqID=`3d619ca1-a2cc-41c7-b765-f9f0895af96f`::Granted request >clientIFinit::DEBUG::2014-10-15 06:44:28,524::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`da4cd80b-9dfa-42e8-bded-10a7c158bba3`::_resourcesAcquired: Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 (shared) >clientIFinit::DEBUG::2014-10-15 06:44:28,524::task::993::Storage.TaskManager.Task::(_decref) Task=`da4cd80b-9dfa-42e8-bded-10a7c158bba3`::ref 1 aborting False >Thread-16::DEBUG::2014-10-15 06:44:28,524::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:28,524::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:28,525::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:28,525::__init__::232::IOProcess::(_processLogs) (22) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:44:28,527::__init__::232::IOProcess::(_processLogs) (22) Queuing response >clientIFinit::DEBUG::2014-10-15 06:44:28,527::fileSD::499::Storage.StorageDomain::(activateVolumes) Fixing permissions on /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb >Thread-16::DEBUG::2014-10-15 06:44:28,527::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:28,527::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:28,528::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:28,528::__init__::232::IOProcess::(_processLogs) (23) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:44:28,528::__init__::232::IOProcess::(_processLogs) (23) Queuing response >clientIFinit::DEBUG::2014-10-15 06:44:28,528::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /var/run/vdsm/storage/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >clientIFinit::WARNING::2014-10-15 06:44:28,528::fileUtils::149::Storage.fileUtils::(createdir) Dir /var/run/vdsm/storage/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 already exists >clientIFinit::DEBUG::2014-10-15 06:44:28,528::fileSD::474::Storage.StorageDomain::(createImageLinks) Creating symlink from /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610 to /var/run/vdsm/storage/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/a5611808-d282-45f9-9771-c3797b521610 >clientIFinit::DEBUG::2014-10-15 06:44:28,528::fileSD::479::Storage.StorageDomain::(createImageLinks) img run dir already exists: /var/run/vdsm/storage/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/a5611808-d282-45f9-9771-c3797b521610 >Thread-16::DEBUG::2014-10-15 06:44:28,529::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:28,529::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:28,529::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:28,529::__init__::232::IOProcess::(_processLogs) (24) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:44:28,529::__init__::232::IOProcess::(_processLogs) (24) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:28,530::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:28,530::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:28,530::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:28,530::__init__::232::IOProcess::(_processLogs) (25) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:28,530::__init__::232::IOProcess::(_processLogs) (25) Queuing response >clientIFinit::DEBUG::2014-10-15 06:44:28,530::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for 486f9fc4-3108-49db-ae60-4dc85544e8eb >Thread-16::DEBUG::2014-10-15 06:44:28,530::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:28,530::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:28,531::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:28,531::__init__::232::IOProcess::(_processLogs) (26) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:28,531::__init__::232::IOProcess::(_processLogs) (26) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:28,531::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:28,531::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:28,531::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:28,531::__init__::232::IOProcess::(_processLogs) (27) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:28,531::__init__::232::IOProcess::(_processLogs) (27) Queuing response >clientIFinit::INFO::2014-10-15 06:44:28,532::logUtils::47::dispatcher::(wrapper) Run and protect: prepareImage, Return response: {'info': {'domainID': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': u'a5611808-d282-45f9-9771-c3797b521610'}, 'path': u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'imgVolumesInfo': [{'domainID': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': u'a5611808-d282-45f9-9771-c3797b521610'}]} >clientIFinit::DEBUG::2014-10-15 06:44:28,532::task::1191::Storage.TaskManager.Task::(prepare) Task=`da4cd80b-9dfa-42e8-bded-10a7c158bba3`::finished: {'info': {'domainID': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': u'a5611808-d282-45f9-9771-c3797b521610'}, 'path': u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'imgVolumesInfo': [{'domainID': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': u'a5611808-d282-45f9-9771-c3797b521610'}]} >clientIFinit::DEBUG::2014-10-15 06:44:28,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`da4cd80b-9dfa-42e8-bded-10a7c158bba3`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 06:44:28,532::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': < ResourceRef 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', isValid: 'True' obj: 'None'>} >clientIFinit::DEBUG::2014-10-15 06:44:28,532::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 06:44:28,532::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' >clientIFinit::DEBUG::2014-10-15 06:44:28,532::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' (0 active users) >clientIFinit::DEBUG::2014-10-15 06:44:28,532::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' is free, finding out if anyone is waiting for it. >clientIFinit::DEBUG::2014-10-15 06:44:28,532::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', Clearing records. >clientIFinit::DEBUG::2014-10-15 06:44:28,533::task::993::Storage.TaskManager.Task::(_decref) Task=`da4cd80b-9dfa-42e8-bded-10a7c158bba3`::ref 0 aborting False >clientIFinit::INFO::2014-10-15 06:44:28,533::clientIF::331::vds::(prepareVolumePath) prepared volume path: /rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:28,926::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:28,926::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-37::DEBUG::2014-10-15 06:44:28,928::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6d94846-f292-45ec-9d0c-37e9dc4e5415`::moving from state init -> state preparing >Thread-37::INFO::2014-10-15 06:44:28,928::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-37::INFO::2014-10-15 06:44:28,928::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000350039', 'lastCheck': '3.3', 'valid': True}} >Thread-37::DEBUG::2014-10-15 06:44:28,928::task::1191::Storage.TaskManager.Task::(prepare) Task=`f6d94846-f292-45ec-9d0c-37e9dc4e5415`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000350039', 'lastCheck': '3.3', 'valid': True}} >Thread-37::DEBUG::2014-10-15 06:44:28,928::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6d94846-f292-45ec-9d0c-37e9dc4e5415`::moving from state preparing -> state finished >Thread-37::DEBUG::2014-10-15 06:44:28,928::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-37::DEBUG::2014-10-15 06:44:28,928::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-37::DEBUG::2014-10-15 06:44:28,929::task::993::Storage.TaskManager.Task::(_decref) Task=`f6d94846-f292-45ec-9d0c-37e9dc4e5415`::ref 0 aborting False >Thread-37::ERROR::2014-10-15 06:44:28,929::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-37::DEBUG::2014-10-15 06:44:28,930::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:28,938::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:28,939::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-38::DEBUG::2014-10-15 06:44:28,940::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-38::DEBUG::2014-10-15 06:44:28,940::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:44:29,400::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:30,402::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:31,403::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:32,043::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:32,044::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-39::DEBUG::2014-10-15 06:44:32,047::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:32,049::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:32,050::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-40::DEBUG::2014-10-15 06:44:32,052::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-40::DEBUG::2014-10-15 06:44:32,054::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:44:32,405::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:32,470::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:32,470::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:32,471::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:44:33,407::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:34,408::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:35,081::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:35,082::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-41::DEBUG::2014-10-15 06:44:35,083::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:35,085::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:35,085::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-42::DEBUG::2014-10-15 06:44:35,086::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-42::DEBUG::2014-10-15 06:44:35,086::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:44:35,409::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:44:35,642::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:35,642::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:35,642::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:35,642::__init__::232::IOProcess::(_processLogs) (28) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:44:35,642::__init__::232::IOProcess::(_processLogs) (28) Queuing response >Thread-25::DEBUG::2014-10-15 06:44:35,643::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:44:35,648::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000356896 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:44:35,649::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:35,649::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:35,649::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:35,650::__init__::232::IOProcess::(_processLogs) (29) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:44:35,650::__init__::232::IOProcess::(_processLogs) (29) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:35,650::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:35,650::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:35,650::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:35,650::__init__::232::IOProcess::(_processLogs) (30) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:35,651::__init__::232::IOProcess::(_processLogs) (30) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:35,651::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:35,651::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:35,651::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:35,651::__init__::232::IOProcess::(_processLogs) (31) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:35,651::__init__::232::IOProcess::(_processLogs) (31) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:35,652::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:35,652::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:35,652::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:35,652::__init__::232::IOProcess::(_processLogs) (32) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:35,652::__init__::232::IOProcess::(_processLogs) (32) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:44:36,410::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:37,412::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:37,412::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:44:37,412::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:37,476::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:37,477::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:37,477::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:38,107::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:38,108::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-43::DEBUG::2014-10-15 06:44:38,109::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:38,110::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:38,111::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-44::DEBUG::2014-10-15 06:44:38,112::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-44::DEBUG::2014-10-15 06:44:38,112::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:44:38,413::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:39,415::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:40,416::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:41,141::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:41,142::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-45::DEBUG::2014-10-15 06:44:41,143::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:41,144::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:41,144::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-46::DEBUG::2014-10-15 06:44:41,145::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-46::DEBUG::2014-10-15 06:44:41,146::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:44:41,417::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:42,418::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:42,483::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:42,483::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:42,484::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:44:43,420::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:44,162::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:44,163::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-47::DEBUG::2014-10-15 06:44:44,164::task::595::Storage.TaskManager.Task::(_updateState) Task=`1439ae9f-4572-44dd-901d-0e444e3b24c1`::moving from state init -> state preparing >Thread-47::INFO::2014-10-15 06:44:44,165::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-10-15 06:44:44,165::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000356896', 'lastCheck': '8.5', 'valid': True}} >Thread-47::DEBUG::2014-10-15 06:44:44,165::task::1191::Storage.TaskManager.Task::(prepare) Task=`1439ae9f-4572-44dd-901d-0e444e3b24c1`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000356896', 'lastCheck': '8.5', 'valid': True}} >Thread-47::DEBUG::2014-10-15 06:44:44,165::task::595::Storage.TaskManager.Task::(_updateState) Task=`1439ae9f-4572-44dd-901d-0e444e3b24c1`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-10-15 06:44:44,165::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-10-15 06:44:44,165::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-10-15 06:44:44,165::task::993::Storage.TaskManager.Task::(_decref) Task=`1439ae9f-4572-44dd-901d-0e444e3b24c1`::ref 0 aborting False >Thread-47::ERROR::2014-10-15 06:44:44,166::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-47::DEBUG::2014-10-15 06:44:44,167::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:44,178::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:44,178::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-48::DEBUG::2014-10-15 06:44:44,179::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-48::DEBUG::2014-10-15 06:44:44,180::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:44:44,421::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:45,422::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:44:45,653::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:45,653::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:45,653::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:45,653::__init__::232::IOProcess::(_processLogs) (33) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:44:45,653::__init__::232::IOProcess::(_processLogs) (33) Queuing response >Thread-25::DEBUG::2014-10-15 06:44:45,654::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:44:45,660::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000393485 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:44:45,660::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:45,660::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:45,661::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:45,661::__init__::232::IOProcess::(_processLogs) (34) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:44:45,661::__init__::232::IOProcess::(_processLogs) (34) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:45,661::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:45,661::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:45,662::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:45,662::__init__::232::IOProcess::(_processLogs) (35) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:45,662::__init__::232::IOProcess::(_processLogs) (35) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:45,662::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:45,662::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:45,662::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:45,662::__init__::232::IOProcess::(_processLogs) (36) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:45,663::__init__::232::IOProcess::(_processLogs) (36) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:45,663::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:45,663::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:45,663::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:45,663::__init__::232::IOProcess::(_processLogs) (37) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:45,663::__init__::232::IOProcess::(_processLogs) (37) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:44:46,423::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:47,294::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:47,295::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-49::DEBUG::2014-10-15 06:44:47,296::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:47,297::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:47,298::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-50::DEBUG::2014-10-15 06:44:47,299::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-50::DEBUG::2014-10-15 06:44:47,299::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:44:47,425::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:47,487::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:47,487::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:47,488::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:44:48,426::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:49,427::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:50,338::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:50,338::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-51::DEBUG::2014-10-15 06:44:50,339::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:50,341::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:50,341::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-52::DEBUG::2014-10-15 06:44:50,342::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-52::DEBUG::2014-10-15 06:44:50,342::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:44:50,429::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:51,430::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:52,431::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:52,493::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:52,494::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:52,494::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:53,360::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:53,361::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-53::DEBUG::2014-10-15 06:44:53,362::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:53,363::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:53,363::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-54::DEBUG::2014-10-15 06:44:53,364::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-54::DEBUG::2014-10-15 06:44:53,365::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:44:53,432::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:54,433::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:55,435::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:44:55,664::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:55,664::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:55,665::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:55,665::__init__::232::IOProcess::(_processLogs) (38) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:44:55,665::__init__::232::IOProcess::(_processLogs) (38) Queuing response >Thread-25::DEBUG::2014-10-15 06:44:55,665::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:44:55,672::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000915831 s, 501 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:44:55,673::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:55,673::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:55,673::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:55,673::__init__::232::IOProcess::(_processLogs) (39) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:44:55,674::__init__::232::IOProcess::(_processLogs) (39) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:55,674::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:55,675::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:55,675::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:55,675::__init__::232::IOProcess::(_processLogs) (40) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:55,675::__init__::232::IOProcess::(_processLogs) (40) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:55,675::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:55,675::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:55,675::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:55,676::__init__::232::IOProcess::(_processLogs) (41) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:55,676::__init__::232::IOProcess::(_processLogs) (41) Queuing response >Thread-16::DEBUG::2014-10-15 06:44:55,676::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:44:55,676::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:44:55,676::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:44:55,676::__init__::232::IOProcess::(_processLogs) (42) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:44:55,676::__init__::232::IOProcess::(_processLogs) (42) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:56,387::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:56,388::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-55::DEBUG::2014-10-15 06:44:56,389::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:56,390::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:56,391::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-56::DEBUG::2014-10-15 06:44:56,392::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-56::DEBUG::2014-10-15 06:44:56,392::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:44:56,436::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:44:57,437::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:57,498::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:57,498::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:44:57,499::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:44:58,439::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:59,415::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:59,415::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-57::DEBUG::2014-10-15 06:44:59,417::task::595::Storage.TaskManager.Task::(_updateState) Task=`ccb8159e-a291-4ff1-b424-707b81826384`::moving from state init -> state preparing >Thread-57::INFO::2014-10-15 06:44:59,417::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-57::INFO::2014-10-15 06:44:59,417::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000915831', 'lastCheck': '3.7', 'valid': True}} >Thread-57::DEBUG::2014-10-15 06:44:59,417::task::1191::Storage.TaskManager.Task::(prepare) Task=`ccb8159e-a291-4ff1-b424-707b81826384`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000915831', 'lastCheck': '3.7', 'valid': True}} >Thread-57::DEBUG::2014-10-15 06:44:59,417::task::595::Storage.TaskManager.Task::(_updateState) Task=`ccb8159e-a291-4ff1-b424-707b81826384`::moving from state preparing -> state finished >Thread-57::DEBUG::2014-10-15 06:44:59,417::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-57::DEBUG::2014-10-15 06:44:59,418::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-57::DEBUG::2014-10-15 06:44:59,418::task::993::Storage.TaskManager.Task::(_decref) Task=`ccb8159e-a291-4ff1-b424-707b81826384`::ref 0 aborting False >Thread-57::ERROR::2014-10-15 06:44:59,419::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-57::DEBUG::2014-10-15 06:44:59,419::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:44:59,429::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:44:59,430::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-58::DEBUG::2014-10-15 06:44:59,431::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-58::DEBUG::2014-10-15 06:44:59,431::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:44:59,439::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:00,441::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:01,442::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:02,444::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:02,504::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:02,505::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:02,505::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:02,545::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:02,545::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-59::DEBUG::2014-10-15 06:45:02,546::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:02,547::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:02,547::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-60::DEBUG::2014-10-15 06:45:02,548::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-60::DEBUG::2014-10-15 06:45:02,548::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-17::DEBUG::2014-10-15 06:45:02,995::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7a0a3ee-279e-45d3-a2a6-67a18543da45`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 06:45:02,995::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:45:02,995::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:02,996::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:02,996::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:02,996::__init__::232::IOProcess::(_processLogs) (43) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:45:02,997::__init__::232::IOProcess::(_processLogs) (43) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:02,997::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:02,998::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:02,998::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:02,998::__init__::232::IOProcess::(_processLogs) (44) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:45:02,998::__init__::232::IOProcess::(_processLogs) (44) Queuing response >Thread-17::INFO::2014-10-15 06:45:02,998::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:45:02,998::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7a0a3ee-279e-45d3-a2a6-67a18543da45`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:45:02,998::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7a0a3ee-279e-45d3-a2a6-67a18543da45`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 06:45:02,998::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 06:45:02,999::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 06:45:02,999::task::993::Storage.TaskManager.Task::(_decref) Task=`c7a0a3ee-279e-45d3-a2a6-67a18543da45`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:45:03,445::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:04,446::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:05,447::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:05,579::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:05,580::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-61::DEBUG::2014-10-15 06:45:05,581::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:05,582::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:05,583::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-62::DEBUG::2014-10-15 06:45:05,584::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-62::DEBUG::2014-10-15 06:45:05,584::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 06:45:05,677::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:05,677::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:05,678::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:05,678::__init__::232::IOProcess::(_processLogs) (45) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:45:05,678::__init__::232::IOProcess::(_processLogs) (45) Queuing response >Thread-25::DEBUG::2014-10-15 06:45:05,678::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:45:05,685::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000382755 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:45:05,685::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:05,685::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:05,686::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:05,686::__init__::232::IOProcess::(_processLogs) (46) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:45:05,686::__init__::232::IOProcess::(_processLogs) (46) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:05,686::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:05,686::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:05,686::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:05,687::__init__::232::IOProcess::(_processLogs) (47) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:05,687::__init__::232::IOProcess::(_processLogs) (47) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:05,687::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:05,687::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:05,687::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:05,687::__init__::232::IOProcess::(_processLogs) (48) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:05,687::__init__::232::IOProcess::(_processLogs) (48) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:05,688::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:05,688::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:05,688::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:05,688::__init__::232::IOProcess::(_processLogs) (49) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:05,688::__init__::232::IOProcess::(_processLogs) (49) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:45:06,449::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:07,450::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:07,450::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:45:07,451::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:07,511::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:07,511::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:07,511::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:45:08,452::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:08,606::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:08,607::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-63::DEBUG::2014-10-15 06:45:08,608::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:08,609::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:08,610::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-64::DEBUG::2014-10-15 06:45:08,611::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-64::DEBUG::2014-10-15 06:45:08,611::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:09,453::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:10,454::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:11,456::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:11,633::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:11,634::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-65::DEBUG::2014-10-15 06:45:11,635::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:11,636::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:11,637::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-66::DEBUG::2014-10-15 06:45:11,637::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-66::DEBUG::2014-10-15 06:45:11,638::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:12,457::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:12,517::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:12,517::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:12,518::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:45:13,458::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:14,459::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:14,660::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:14,661::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-67::DEBUG::2014-10-15 06:45:14,662::task::595::Storage.TaskManager.Task::(_updateState) Task=`7f48d462-70a8-491a-bc23-8112ec30f6b6`::moving from state init -> state preparing >Thread-67::INFO::2014-10-15 06:45:14,662::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-67::INFO::2014-10-15 06:45:14,662::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000382755', 'lastCheck': '9.0', 'valid': True}} >Thread-67::DEBUG::2014-10-15 06:45:14,662::task::1191::Storage.TaskManager.Task::(prepare) Task=`7f48d462-70a8-491a-bc23-8112ec30f6b6`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000382755', 'lastCheck': '9.0', 'valid': True}} >Thread-67::DEBUG::2014-10-15 06:45:14,662::task::595::Storage.TaskManager.Task::(_updateState) Task=`7f48d462-70a8-491a-bc23-8112ec30f6b6`::moving from state preparing -> state finished >Thread-67::DEBUG::2014-10-15 06:45:14,662::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-67::DEBUG::2014-10-15 06:45:14,663::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-67::DEBUG::2014-10-15 06:45:14,663::task::993::Storage.TaskManager.Task::(_decref) Task=`7f48d462-70a8-491a-bc23-8112ec30f6b6`::ref 0 aborting False >Thread-67::ERROR::2014-10-15 06:45:14,663::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-67::DEBUG::2014-10-15 06:45:14,664::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:14,677::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:14,677::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-68::DEBUG::2014-10-15 06:45:14,678::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-68::DEBUG::2014-10-15 06:45:14,679::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:15,460::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:45:15,689::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:15,689::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:15,690::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:15,690::__init__::232::IOProcess::(_processLogs) (50) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:45:15,690::__init__::232::IOProcess::(_processLogs) (50) Queuing response >Thread-25::DEBUG::2014-10-15 06:45:15,690::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:45:15,696::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000441958 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:45:15,697::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:15,697::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:15,697::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:15,697::__init__::232::IOProcess::(_processLogs) (51) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:45:15,698::__init__::232::IOProcess::(_processLogs) (51) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:15,698::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:15,698::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:15,698::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:15,698::__init__::232::IOProcess::(_processLogs) (52) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:15,698::__init__::232::IOProcess::(_processLogs) (52) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:15,699::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:15,699::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:15,699::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:15,699::__init__::232::IOProcess::(_processLogs) (53) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:15,699::__init__::232::IOProcess::(_processLogs) (53) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:15,700::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:15,700::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:15,700::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:15,700::__init__::232::IOProcess::(_processLogs) (54) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:15,700::__init__::232::IOProcess::(_processLogs) (54) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:45:16,461::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:17,463::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:17,524::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:17,524::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:17,525::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:17,798::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:17,799::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-69::DEBUG::2014-10-15 06:45:17,800::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:17,801::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:17,802::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-70::DEBUG::2014-10-15 06:45:17,803::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-70::DEBUG::2014-10-15 06:45:17,803::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:18,464::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:19,465::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:20,467::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:20,854::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:20,855::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-71::DEBUG::2014-10-15 06:45:20,856::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:20,857::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:20,857::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-72::DEBUG::2014-10-15 06:45:20,858::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-72::DEBUG::2014-10-15 06:45:20,859::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:21,468::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:22,469::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:22,530::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:22,531::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:22,531::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:45:23,470::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:23,893::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:23,893::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-73::DEBUG::2014-10-15 06:45:23,894::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:23,896::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:23,896::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-74::DEBUG::2014-10-15 06:45:23,897::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-74::DEBUG::2014-10-15 06:45:23,897::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:24,471::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:25,473::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:45:25,701::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:25,701::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:25,701::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:25,701::__init__::232::IOProcess::(_processLogs) (55) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:45:25,702::__init__::232::IOProcess::(_processLogs) (55) Queuing response >Thread-25::DEBUG::2014-10-15 06:45:25,702::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:45:25,708::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000355513 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:45:25,709::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:25,709::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:25,709::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:25,709::__init__::232::IOProcess::(_processLogs) (56) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:45:25,709::__init__::232::IOProcess::(_processLogs) (56) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:25,710::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:25,710::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:25,710::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:25,710::__init__::232::IOProcess::(_processLogs) (57) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:25,710::__init__::232::IOProcess::(_processLogs) (57) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:25,710::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:25,711::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:25,711::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:25,711::__init__::232::IOProcess::(_processLogs) (58) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:25,711::__init__::232::IOProcess::(_processLogs) (58) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:25,711::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:25,711::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:25,712::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:25,712::__init__::232::IOProcess::(_processLogs) (59) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:25,712::__init__::232::IOProcess::(_processLogs) (59) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:45:26,474::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:26,918::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:26,919::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-75::DEBUG::2014-10-15 06:45:26,920::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:26,921::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:26,922::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-76::DEBUG::2014-10-15 06:45:26,923::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-76::DEBUG::2014-10-15 06:45:26,923::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:27,475::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:27,537::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:27,537::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:27,538::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:45:28,476::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:29,478::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:29,960::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:29,960::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-77::DEBUG::2014-10-15 06:45:29,962::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f07b8a8-5556-4863-91cf-27683fb32521`::moving from state init -> state preparing >Thread-77::INFO::2014-10-15 06:45:29,962::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-77::INFO::2014-10-15 06:45:29,962::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000355513', 'lastCheck': '4.2', 'valid': True}} >Thread-77::DEBUG::2014-10-15 06:45:29,962::task::1191::Storage.TaskManager.Task::(prepare) Task=`3f07b8a8-5556-4863-91cf-27683fb32521`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000355513', 'lastCheck': '4.2', 'valid': True}} >Thread-77::DEBUG::2014-10-15 06:45:29,962::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f07b8a8-5556-4863-91cf-27683fb32521`::moving from state preparing -> state finished >Thread-77::DEBUG::2014-10-15 06:45:29,962::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-77::DEBUG::2014-10-15 06:45:29,962::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-77::DEBUG::2014-10-15 06:45:29,962::task::993::Storage.TaskManager.Task::(_decref) Task=`3f07b8a8-5556-4863-91cf-27683fb32521`::ref 0 aborting False >Thread-77::ERROR::2014-10-15 06:45:29,963::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-77::DEBUG::2014-10-15 06:45:29,964::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:29,973::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:29,973::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-78::DEBUG::2014-10-15 06:45:29,974::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-78::DEBUG::2014-10-15 06:45:29,975::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:30,479::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:31,480::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:32,481::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:32,543::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:32,544::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:32,544::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:33,112::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:33,112::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-79::DEBUG::2014-10-15 06:45:33,113::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:33,115::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:33,115::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-80::DEBUG::2014-10-15 06:45:33,116::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-80::DEBUG::2014-10-15 06:45:33,116::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:33,482::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:34,484::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:35,485::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:45:35,713::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:35,713::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:35,713::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:35,713::__init__::232::IOProcess::(_processLogs) (60) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:45:35,713::__init__::232::IOProcess::(_processLogs) (60) Queuing response >Thread-25::DEBUG::2014-10-15 06:45:35,714::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:45:35,720::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.0003843 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:45:35,720::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:35,720::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:35,721::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:35,721::__init__::232::IOProcess::(_processLogs) (61) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:45:35,721::__init__::232::IOProcess::(_processLogs) (61) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:35,721::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:35,722::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:35,722::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:35,722::__init__::232::IOProcess::(_processLogs) (62) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:35,722::__init__::232::IOProcess::(_processLogs) (62) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:35,722::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:35,722::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:35,722::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:35,723::__init__::232::IOProcess::(_processLogs) (63) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:35,723::__init__::232::IOProcess::(_processLogs) (63) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:35,723::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:35,723::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:35,723::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:35,723::__init__::232::IOProcess::(_processLogs) (64) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:35,723::__init__::232::IOProcess::(_processLogs) (64) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:36,147::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:36,147::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-81::DEBUG::2014-10-15 06:45:36,149::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:36,150::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:36,150::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-82::DEBUG::2014-10-15 06:45:36,151::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-82::DEBUG::2014-10-15 06:45:36,152::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:36,486::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:37,487::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:37,487::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:45:37,488::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:37,550::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:37,550::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:37,551::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:45:38,489::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:39,171::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:39,172::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-83::DEBUG::2014-10-15 06:45:39,173::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:39,174::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:39,174::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-84::DEBUG::2014-10-15 06:45:39,175::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-84::DEBUG::2014-10-15 06:45:39,176::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:39,490::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:40,492::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:41,493::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:42,217::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:42,218::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-85::DEBUG::2014-10-15 06:45:42,219::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:42,220::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:42,220::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-86::DEBUG::2014-10-15 06:45:42,221::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-86::DEBUG::2014-10-15 06:45:42,221::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:42,494::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:42,556::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:42,557::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:42,557::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:45:43,495::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:44,496::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:45,240::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:45,241::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-87::DEBUG::2014-10-15 06:45:45,242::task::595::Storage.TaskManager.Task::(_updateState) Task=`c0aee348-2e22-43f8-ae29-2df4737fb3ce`::moving from state init -> state preparing >Thread-87::INFO::2014-10-15 06:45:45,242::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-87::INFO::2014-10-15 06:45:45,242::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0003843', 'lastCheck': '9.5', 'valid': True}} >Thread-87::DEBUG::2014-10-15 06:45:45,242::task::1191::Storage.TaskManager.Task::(prepare) Task=`c0aee348-2e22-43f8-ae29-2df4737fb3ce`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0003843', 'lastCheck': '9.5', 'valid': True}} >Thread-87::DEBUG::2014-10-15 06:45:45,243::task::595::Storage.TaskManager.Task::(_updateState) Task=`c0aee348-2e22-43f8-ae29-2df4737fb3ce`::moving from state preparing -> state finished >Thread-87::DEBUG::2014-10-15 06:45:45,243::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-87::DEBUG::2014-10-15 06:45:45,243::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-87::DEBUG::2014-10-15 06:45:45,243::task::993::Storage.TaskManager.Task::(_decref) Task=`c0aee348-2e22-43f8-ae29-2df4737fb3ce`::ref 0 aborting False >Thread-87::ERROR::2014-10-15 06:45:45,244::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-87::DEBUG::2014-10-15 06:45:45,244::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:45,253::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:45,254::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-88::DEBUG::2014-10-15 06:45:45,255::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-88::DEBUG::2014-10-15 06:45:45,255::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:45,498::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:45:45,724::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:45,725::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:45,725::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:45,725::__init__::232::IOProcess::(_processLogs) (65) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:45:45,725::__init__::232::IOProcess::(_processLogs) (65) Queuing response >Thread-25::DEBUG::2014-10-15 06:45:45,725::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:45:45,731::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000445131 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:45:45,732::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:45,732::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:45,732::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:45,732::__init__::232::IOProcess::(_processLogs) (66) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:45:45,732::__init__::232::IOProcess::(_processLogs) (66) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:45,733::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:45,733::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:45,733::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:45,733::__init__::232::IOProcess::(_processLogs) (67) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:45,733::__init__::232::IOProcess::(_processLogs) (67) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:45,734::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:45,734::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:45,734::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:45,734::__init__::232::IOProcess::(_processLogs) (68) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:45,734::__init__::232::IOProcess::(_processLogs) (68) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:45,734::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:45,735::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:45,735::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:45,735::__init__::232::IOProcess::(_processLogs) (69) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:45,735::__init__::232::IOProcess::(_processLogs) (69) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:45:46,499::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:47,500::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:47,563::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:47,563::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:47,564::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:48,366::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:48,367::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-89::DEBUG::2014-10-15 06:45:48,368::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:48,369::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:48,370::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-90::DEBUG::2014-10-15 06:45:48,371::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-90::DEBUG::2014-10-15 06:45:48,371::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:48,501::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:49,503::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:50,504::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:51,410::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:51,410::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-91::DEBUG::2014-10-15 06:45:51,411::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:51,412::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:51,413::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-92::DEBUG::2014-10-15 06:45:51,414::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-92::DEBUG::2014-10-15 06:45:51,414::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:51,505::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:52,506::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:52,569::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:52,570::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:52,570::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:45:53,508::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:54,438::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:54,439::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-93::DEBUG::2014-10-15 06:45:54,440::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:54,441::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:54,442::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-94::DEBUG::2014-10-15 06:45:54,443::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-94::DEBUG::2014-10-15 06:45:54,443::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:54,509::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:55,510::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:45:55,736::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:55,736::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:55,736::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:55,736::__init__::232::IOProcess::(_processLogs) (70) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:45:55,736::__init__::232::IOProcess::(_processLogs) (70) Queuing response >Thread-25::DEBUG::2014-10-15 06:45:55,737::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:45:55,743::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000360097 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:45:55,743::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:55,743::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:55,743::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:55,744::__init__::232::IOProcess::(_processLogs) (71) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:45:55,744::__init__::232::IOProcess::(_processLogs) (71) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:55,744::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:55,744::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:55,745::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:55,745::__init__::232::IOProcess::(_processLogs) (72) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:55,745::__init__::232::IOProcess::(_processLogs) (72) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:55,745::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:55,745::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:55,745::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:55,745::__init__::232::IOProcess::(_processLogs) (73) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:55,746::__init__::232::IOProcess::(_processLogs) (73) Queuing response >Thread-16::DEBUG::2014-10-15 06:45:55,746::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:45:55,746::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:45:55,746::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:45:55,746::__init__::232::IOProcess::(_processLogs) (74) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:45:55,746::__init__::232::IOProcess::(_processLogs) (74) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:45:56,511::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:57,466::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:57,466::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-95::DEBUG::2014-10-15 06:45:57,467::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:45:57,468::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:45:57,469::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-96::DEBUG::2014-10-15 06:45:57,470::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-96::DEBUG::2014-10-15 06:45:57,470::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:45:57,513::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:57,576::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:57,576::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:45:57,577::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:45:58,514::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:45:59,515::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:00,491::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:00,492::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-97::DEBUG::2014-10-15 06:46:00,493::task::595::Storage.TaskManager.Task::(_updateState) Task=`a7936fea-bec4-4e7c-9859-54d9dd582faf`::moving from state init -> state preparing >Thread-97::INFO::2014-10-15 06:46:00,493::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-10-15 06:46:00,494::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000360097', 'lastCheck': '4.7', 'valid': True}} >Thread-97::DEBUG::2014-10-15 06:46:00,494::task::1191::Storage.TaskManager.Task::(prepare) Task=`a7936fea-bec4-4e7c-9859-54d9dd582faf`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000360097', 'lastCheck': '4.7', 'valid': True}} >Thread-97::DEBUG::2014-10-15 06:46:00,494::task::595::Storage.TaskManager.Task::(_updateState) Task=`a7936fea-bec4-4e7c-9859-54d9dd582faf`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-10-15 06:46:00,494::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-10-15 06:46:00,494::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-10-15 06:46:00,494::task::993::Storage.TaskManager.Task::(_decref) Task=`a7936fea-bec4-4e7c-9859-54d9dd582faf`::ref 0 aborting False >Thread-97::ERROR::2014-10-15 06:46:00,495::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-97::DEBUG::2014-10-15 06:46:00,496::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:00,508::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:00,509::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-98::DEBUG::2014-10-15 06:46:00,510::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-98::DEBUG::2014-10-15 06:46:00,510::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:00,516::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:01,518::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:02,519::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:02,582::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:02,583::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:02,583::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 06:46:03,037::task::595::Storage.TaskManager.Task::(_updateState) Task=`c84e1d47-8e68-4045-a572-b5d08da4d7d8`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 06:46:03,038::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:46:03,038::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:03,038::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:03,038::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:03,038::__init__::232::IOProcess::(_processLogs) (75) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:46:03,040::__init__::232::IOProcess::(_processLogs) (75) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:03,041::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:03,041::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:03,041::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:03,041::__init__::232::IOProcess::(_processLogs) (76) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:46:03,041::__init__::232::IOProcess::(_processLogs) (76) Queuing response >Thread-17::INFO::2014-10-15 06:46:03,041::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:46:03,041::task::1191::Storage.TaskManager.Task::(prepare) Task=`c84e1d47-8e68-4045-a572-b5d08da4d7d8`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:46:03,042::task::595::Storage.TaskManager.Task::(_updateState) Task=`c84e1d47-8e68-4045-a572-b5d08da4d7d8`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 06:46:03,042::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 06:46:03,042::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 06:46:03,042::task::993::Storage.TaskManager.Task::(_decref) Task=`c84e1d47-8e68-4045-a572-b5d08da4d7d8`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:46:03,521::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:03,607::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:03,608::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-99::DEBUG::2014-10-15 06:46:03,609::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:03,610::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:03,611::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-100::DEBUG::2014-10-15 06:46:03,611::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-100::DEBUG::2014-10-15 06:46:03,612::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:04,522::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:05,523::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:46:05,747::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:05,747::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:05,747::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:05,748::__init__::232::IOProcess::(_processLogs) (77) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:46:05,748::__init__::232::IOProcess::(_processLogs) (77) Queuing response >Thread-25::DEBUG::2014-10-15 06:46:05,748::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:46:05,755::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000432786 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:46:05,755::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:05,756::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:05,756::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:05,756::__init__::232::IOProcess::(_processLogs) (78) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:46:05,756::__init__::232::IOProcess::(_processLogs) (78) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:05,757::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:05,757::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:05,757::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:05,757::__init__::232::IOProcess::(_processLogs) (79) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:05,757::__init__::232::IOProcess::(_processLogs) (79) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:05,757::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:05,757::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:05,757::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:05,758::__init__::232::IOProcess::(_processLogs) (80) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:05,758::__init__::232::IOProcess::(_processLogs) (80) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:05,758::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:05,758::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:05,758::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:05,758::__init__::232::IOProcess::(_processLogs) (81) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:05,758::__init__::232::IOProcess::(_processLogs) (81) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:46:06,524::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:06,654::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:06,655::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-101::DEBUG::2014-10-15 06:46:06,656::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:06,657::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:06,657::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-102::DEBUG::2014-10-15 06:46:06,658::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-102::DEBUG::2014-10-15 06:46:06,659::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:07,525::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:07,526::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:46:07,526::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:07,589::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:07,589::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:07,590::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:46:08,527::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:09,528::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:09,678::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:09,678::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-103::DEBUG::2014-10-15 06:46:09,679::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:09,681::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:09,681::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-104::DEBUG::2014-10-15 06:46:09,682::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-104::DEBUG::2014-10-15 06:46:09,683::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:10,530::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:11,531::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:12,532::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:12,595::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:12,596::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:12,596::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:12,705::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:12,705::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-105::DEBUG::2014-10-15 06:46:12,706::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:12,708::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:12,708::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-106::DEBUG::2014-10-15 06:46:12,709::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-106::DEBUG::2014-10-15 06:46:12,709::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:13,534::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:14,535::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:15,536::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:15,732::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:15,733::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-107::DEBUG::2014-10-15 06:46:15,734::task::595::Storage.TaskManager.Task::(_updateState) Task=`609235aa-7f7d-43fd-89b7-67e7f152d9bb`::moving from state init -> state preparing >Thread-107::INFO::2014-10-15 06:46:15,734::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-107::INFO::2014-10-15 06:46:15,735::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432786', 'lastCheck': '10.0', 'valid': True}} >Thread-107::DEBUG::2014-10-15 06:46:15,735::task::1191::Storage.TaskManager.Task::(prepare) Task=`609235aa-7f7d-43fd-89b7-67e7f152d9bb`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432786', 'lastCheck': '10.0', 'valid': True}} >Thread-107::DEBUG::2014-10-15 06:46:15,735::task::595::Storage.TaskManager.Task::(_updateState) Task=`609235aa-7f7d-43fd-89b7-67e7f152d9bb`::moving from state preparing -> state finished >Thread-107::DEBUG::2014-10-15 06:46:15,735::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-107::DEBUG::2014-10-15 06:46:15,735::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-107::DEBUG::2014-10-15 06:46:15,735::task::993::Storage.TaskManager.Task::(_decref) Task=`609235aa-7f7d-43fd-89b7-67e7f152d9bb`::ref 0 aborting False >Thread-107::ERROR::2014-10-15 06:46:15,736::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-107::DEBUG::2014-10-15 06:46:15,737::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:15,748::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:15,749::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-108::DEBUG::2014-10-15 06:46:15,750::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-108::DEBUG::2014-10-15 06:46:15,750::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 06:46:15,759::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:15,759::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:15,760::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:15,760::__init__::232::IOProcess::(_processLogs) (82) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:46:15,760::__init__::232::IOProcess::(_processLogs) (82) Queuing response >Thread-25::DEBUG::2014-10-15 06:46:15,760::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:46:15,766::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00039174 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:46:15,767::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:15,767::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:15,767::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:15,767::__init__::232::IOProcess::(_processLogs) (83) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:46:15,767::__init__::232::IOProcess::(_processLogs) (83) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:15,768::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:15,768::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:15,768::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:15,768::__init__::232::IOProcess::(_processLogs) (84) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:15,768::__init__::232::IOProcess::(_processLogs) (84) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:15,768::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:15,769::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:15,769::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:15,769::__init__::232::IOProcess::(_processLogs) (85) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:15,769::__init__::232::IOProcess::(_processLogs) (85) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:15,769::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:15,769::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:15,769::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:15,770::__init__::232::IOProcess::(_processLogs) (86) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:15,770::__init__::232::IOProcess::(_processLogs) (86) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:46:16,537::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:17,539::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:17,598::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:17,598::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:17,599::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:46:18,540::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:18,863::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:18,863::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-109::DEBUG::2014-10-15 06:46:18,864::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:18,866::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:18,866::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-110::DEBUG::2014-10-15 06:46:18,867::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-110::DEBUG::2014-10-15 06:46:18,867::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:19,541::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:20,542::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:21,544::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:21,896::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:21,896::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-111::DEBUG::2014-10-15 06:46:21,897::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:21,898::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:21,899::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-112::DEBUG::2014-10-15 06:46:21,900::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-112::DEBUG::2014-10-15 06:46:21,901::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:22,545::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:22,604::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:22,604::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:22,605::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:46:23,546::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:24,547::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:24,918::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:24,919::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-113::DEBUG::2014-10-15 06:46:24,920::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:24,921::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:24,921::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-114::DEBUG::2014-10-15 06:46:24,922::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-114::DEBUG::2014-10-15 06:46:24,923::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:25,549::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:46:25,771::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:25,771::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:25,771::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:25,771::__init__::232::IOProcess::(_processLogs) (87) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:46:25,771::__init__::232::IOProcess::(_processLogs) (87) Queuing response >Thread-25::DEBUG::2014-10-15 06:46:25,772::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:46:25,778::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000368712 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:46:25,778::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:25,779::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:25,779::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:25,779::__init__::232::IOProcess::(_processLogs) (88) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:46:25,779::__init__::232::IOProcess::(_processLogs) (88) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:25,779::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:25,780::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:25,780::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:25,780::__init__::232::IOProcess::(_processLogs) (89) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:25,780::__init__::232::IOProcess::(_processLogs) (89) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:25,780::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:25,780::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:25,780::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:25,780::__init__::232::IOProcess::(_processLogs) (90) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:25,781::__init__::232::IOProcess::(_processLogs) (90) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:25,781::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:25,781::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:25,781::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:25,781::__init__::232::IOProcess::(_processLogs) (91) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:25,781::__init__::232::IOProcess::(_processLogs) (91) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:46:26,550::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:27,551::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:27,606::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:27,606::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:27,607::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:27,945::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:27,945::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-115::DEBUG::2014-10-15 06:46:27,947::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:27,948::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:27,948::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-116::DEBUG::2014-10-15 06:46:27,949::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-116::DEBUG::2014-10-15 06:46:27,950::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:28,552::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:29,554::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:30,555::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:30,972::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:30,973::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-117::DEBUG::2014-10-15 06:46:30,974::task::595::Storage.TaskManager.Task::(_updateState) Task=`ffef94b9-a57e-4423-a928-42db2939fac1`::moving from state init -> state preparing >Thread-117::INFO::2014-10-15 06:46:30,975::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-117::INFO::2014-10-15 06:46:30,975::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000368712', 'lastCheck': '5.2', 'valid': True}} >Thread-117::DEBUG::2014-10-15 06:46:30,975::task::1191::Storage.TaskManager.Task::(prepare) Task=`ffef94b9-a57e-4423-a928-42db2939fac1`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000368712', 'lastCheck': '5.2', 'valid': True}} >Thread-117::DEBUG::2014-10-15 06:46:30,975::task::595::Storage.TaskManager.Task::(_updateState) Task=`ffef94b9-a57e-4423-a928-42db2939fac1`::moving from state preparing -> state finished >Thread-117::DEBUG::2014-10-15 06:46:30,975::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-117::DEBUG::2014-10-15 06:46:30,975::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-117::DEBUG::2014-10-15 06:46:30,975::task::993::Storage.TaskManager.Task::(_decref) Task=`ffef94b9-a57e-4423-a928-42db2939fac1`::ref 0 aborting False >Thread-117::ERROR::2014-10-15 06:46:30,976::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-117::DEBUG::2014-10-15 06:46:30,977::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:30,989::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:30,989::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-118::DEBUG::2014-10-15 06:46:30,990::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-118::DEBUG::2014-10-15 06:46:30,991::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:31,556::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:32,558::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:32,611::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:32,611::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:32,612::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:46:33,559::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:34,112::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:34,112::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-119::DEBUG::2014-10-15 06:46:34,113::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:34,114::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:34,115::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-120::DEBUG::2014-10-15 06:46:34,116::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-120::DEBUG::2014-10-15 06:46:34,116::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:34,560::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:35,561::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:46:35,782::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:35,783::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:35,783::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:35,783::__init__::232::IOProcess::(_processLogs) (92) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:46:35,783::__init__::232::IOProcess::(_processLogs) (92) Queuing response >Thread-25::DEBUG::2014-10-15 06:46:35,783::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:46:35,789::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000435961 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:46:35,790::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:35,790::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:35,790::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:35,790::__init__::232::IOProcess::(_processLogs) (93) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:46:35,791::__init__::232::IOProcess::(_processLogs) (93) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:35,791::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:35,791::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:35,791::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:35,792::__init__::232::IOProcess::(_processLogs) (94) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:35,792::__init__::232::IOProcess::(_processLogs) (94) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:35,792::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:35,792::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:35,792::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:35,792::__init__::232::IOProcess::(_processLogs) (95) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:35,792::__init__::232::IOProcess::(_processLogs) (95) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:35,793::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:35,793::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:35,793::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:35,793::__init__::232::IOProcess::(_processLogs) (96) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:35,793::__init__::232::IOProcess::(_processLogs) (96) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:46:36,563::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:37,164::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:37,164::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-121::DEBUG::2014-10-15 06:46:37,166::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:37,167::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:37,167::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-122::DEBUG::2014-10-15 06:46:37,168::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-122::DEBUG::2014-10-15 06:46:37,168::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:37,564::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:37,564::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:46:37,564::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:37,618::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:37,618::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:37,618::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:46:38,565::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:39,567::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:40,187::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:40,188::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-123::DEBUG::2014-10-15 06:46:40,189::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:40,190::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:40,191::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-124::DEBUG::2014-10-15 06:46:40,192::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-124::DEBUG::2014-10-15 06:46:40,192::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:40,568::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:41,569::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:42,571::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:42,624::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:42,625::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:42,625::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:43,213::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:43,214::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-125::DEBUG::2014-10-15 06:46:43,215::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:43,216::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:43,216::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-126::DEBUG::2014-10-15 06:46:43,217::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-126::DEBUG::2014-10-15 06:46:43,218::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:43,572::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:44,573::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:45,574::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:46:45,794::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:45,794::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:45,794::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:45,794::__init__::232::IOProcess::(_processLogs) (97) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:46:45,795::__init__::232::IOProcess::(_processLogs) (97) Queuing response >Thread-25::DEBUG::2014-10-15 06:46:45,795::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:46:45,801::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000401923 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:46:45,802::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:45,802::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:45,802::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:45,802::__init__::232::IOProcess::(_processLogs) (98) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:46:45,802::__init__::232::IOProcess::(_processLogs) (98) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:45,803::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:45,803::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:45,803::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:45,803::__init__::232::IOProcess::(_processLogs) (99) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:45,803::__init__::232::IOProcess::(_processLogs) (99) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:45,803::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:45,803::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:45,804::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:45,804::__init__::232::IOProcess::(_processLogs) (100) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:45,804::__init__::232::IOProcess::(_processLogs) (100) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:45,804::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:45,804::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:45,804::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:45,804::__init__::232::IOProcess::(_processLogs) (101) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:45,805::__init__::232::IOProcess::(_processLogs) (101) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:46,236::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:46,236::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-127::DEBUG::2014-10-15 06:46:46,237::task::595::Storage.TaskManager.Task::(_updateState) Task=`04c065bb-25fa-432d-8522-acb7a22d57ab`::moving from state init -> state preparing >Thread-127::INFO::2014-10-15 06:46:46,238::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-127::INFO::2014-10-15 06:46:46,238::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000401923', 'lastCheck': '0.4', 'valid': True}} >Thread-127::DEBUG::2014-10-15 06:46:46,238::task::1191::Storage.TaskManager.Task::(prepare) Task=`04c065bb-25fa-432d-8522-acb7a22d57ab`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000401923', 'lastCheck': '0.4', 'valid': True}} >Thread-127::DEBUG::2014-10-15 06:46:46,238::task::595::Storage.TaskManager.Task::(_updateState) Task=`04c065bb-25fa-432d-8522-acb7a22d57ab`::moving from state preparing -> state finished >Thread-127::DEBUG::2014-10-15 06:46:46,238::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-127::DEBUG::2014-10-15 06:46:46,238::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-127::DEBUG::2014-10-15 06:46:46,238::task::993::Storage.TaskManager.Task::(_decref) Task=`04c065bb-25fa-432d-8522-acb7a22d57ab`::ref 0 aborting False >Thread-127::ERROR::2014-10-15 06:46:46,239::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-127::DEBUG::2014-10-15 06:46:46,240::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:46,252::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:46,252::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-128::DEBUG::2014-10-15 06:46:46,254::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-128::DEBUG::2014-10-15 06:46:46,254::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:46,576::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:47,577::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:47,631::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:47,631::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:47,632::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:46:48,578::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:49,369::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:49,370::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-129::DEBUG::2014-10-15 06:46:49,371::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:49,373::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:49,373::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-130::DEBUG::2014-10-15 06:46:49,374::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-130::DEBUG::2014-10-15 06:46:49,375::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:49,579::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:50,581::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:51,582::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:52,413::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:52,414::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-131::DEBUG::2014-10-15 06:46:52,415::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:52,417::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:52,418::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-132::DEBUG::2014-10-15 06:46:52,419::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-132::DEBUG::2014-10-15 06:46:52,419::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:52,583::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:52,637::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:52,638::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:52,638::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:46:53,584::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:54,586::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:55,438::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:55,439::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-133::DEBUG::2014-10-15 06:46:55,440::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:55,441::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:55,441::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-134::DEBUG::2014-10-15 06:46:55,442::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-134::DEBUG::2014-10-15 06:46:55,442::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:55,587::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:46:55,805::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:55,806::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:55,806::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:55,806::__init__::232::IOProcess::(_processLogs) (102) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:46:55,806::__init__::232::IOProcess::(_processLogs) (102) Queuing response >Thread-25::DEBUG::2014-10-15 06:46:55,806::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:46:55,812::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00035285 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:46:55,813::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:55,813::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:55,813::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:55,813::__init__::232::IOProcess::(_processLogs) (103) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:46:55,814::__init__::232::IOProcess::(_processLogs) (103) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:55,814::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:55,814::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:55,814::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:55,814::__init__::232::IOProcess::(_processLogs) (104) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:55,815::__init__::232::IOProcess::(_processLogs) (104) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:55,815::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:55,815::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:55,815::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:55,815::__init__::232::IOProcess::(_processLogs) (105) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:55,815::__init__::232::IOProcess::(_processLogs) (105) Queuing response >Thread-16::DEBUG::2014-10-15 06:46:55,816::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:46:55,816::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:46:55,816::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:46:55,816::__init__::232::IOProcess::(_processLogs) (106) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:46:55,816::__init__::232::IOProcess::(_processLogs) (106) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:46:56,588::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:57,589::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:57,644::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:57,644::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:46:57,645::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:58,464::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:58,465::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-135::DEBUG::2014-10-15 06:46:58,466::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:46:58,467::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:46:58,467::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-136::DEBUG::2014-10-15 06:46:58,468::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-136::DEBUG::2014-10-15 06:46:58,469::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:46:58,591::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:46:59,592::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:00,593::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:01,486::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:01,487::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-137::DEBUG::2014-10-15 06:47:01,490::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ac2e7ee-e18f-4dcd-86e0-edfc6ad2d4c3`::moving from state init -> state preparing >Thread-137::INFO::2014-10-15 06:47:01,491::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-137::INFO::2014-10-15 06:47:01,492::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00035285', 'lastCheck': '5.7', 'valid': True}} >Thread-137::DEBUG::2014-10-15 06:47:01,492::task::1191::Storage.TaskManager.Task::(prepare) Task=`3ac2e7ee-e18f-4dcd-86e0-edfc6ad2d4c3`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00035285', 'lastCheck': '5.7', 'valid': True}} >Thread-137::DEBUG::2014-10-15 06:47:01,493::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ac2e7ee-e18f-4dcd-86e0-edfc6ad2d4c3`::moving from state preparing -> state finished >Thread-137::DEBUG::2014-10-15 06:47:01,493::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-137::DEBUG::2014-10-15 06:47:01,494::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-137::DEBUG::2014-10-15 06:47:01,494::task::993::Storage.TaskManager.Task::(_decref) Task=`3ac2e7ee-e18f-4dcd-86e0-edfc6ad2d4c3`::ref 0 aborting False >Thread-137::ERROR::2014-10-15 06:47:01,497::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-137::DEBUG::2014-10-15 06:47:01,499::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:01,513::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:01,514::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-138::DEBUG::2014-10-15 06:47:01,517::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-138::DEBUG::2014-10-15 06:47:01,518::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:01,594::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:02,596::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:02,650::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:02,651::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:02,651::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 06:47:03,084::task::595::Storage.TaskManager.Task::(_updateState) Task=`3c5d4728-9763-46e3-ad4a-13c76b653d2a`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 06:47:03,084::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:47:03,084::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:03,085::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:03,085::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:03,085::__init__::232::IOProcess::(_processLogs) (107) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:47:03,086::__init__::232::IOProcess::(_processLogs) (107) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:03,087::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:03,087::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:03,087::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:03,087::__init__::232::IOProcess::(_processLogs) (108) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:47:03,087::__init__::232::IOProcess::(_processLogs) (108) Queuing response >Thread-17::INFO::2014-10-15 06:47:03,087::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:47:03,087::task::1191::Storage.TaskManager.Task::(prepare) Task=`3c5d4728-9763-46e3-ad4a-13c76b653d2a`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:47:03,087::task::595::Storage.TaskManager.Task::(_updateState) Task=`3c5d4728-9763-46e3-ad4a-13c76b653d2a`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 06:47:03,087::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 06:47:03,087::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 06:47:03,088::task::993::Storage.TaskManager.Task::(_decref) Task=`3c5d4728-9763-46e3-ad4a-13c76b653d2a`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:47:03,597::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:04,598::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:04,625::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:04,626::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-139::DEBUG::2014-10-15 06:47:04,627::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:04,628::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:04,628::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-140::DEBUG::2014-10-15 06:47:04,629::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-140::DEBUG::2014-10-15 06:47:04,629::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:05,599::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:47:05,817::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:05,817::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:05,817::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:05,817::__init__::232::IOProcess::(_processLogs) (109) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:47:05,818::__init__::232::IOProcess::(_processLogs) (109) Queuing response >Thread-25::DEBUG::2014-10-15 06:47:05,818::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:47:05,825::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00047512 s, 966 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:47:05,825::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:05,826::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:05,826::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:05,826::__init__::232::IOProcess::(_processLogs) (110) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:47:05,826::__init__::232::IOProcess::(_processLogs) (110) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:05,826::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:05,827::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:05,827::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:05,827::__init__::232::IOProcess::(_processLogs) (111) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:05,827::__init__::232::IOProcess::(_processLogs) (111) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:05,827::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:05,827::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:05,827::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:05,827::__init__::232::IOProcess::(_processLogs) (112) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:05,828::__init__::232::IOProcess::(_processLogs) (112) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:05,828::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:05,828::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:05,828::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:05,828::__init__::232::IOProcess::(_processLogs) (113) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:05,828::__init__::232::IOProcess::(_processLogs) (113) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:47:06,600::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:07,601::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:07,601::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:47:07,601::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:07,654::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:07,654::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:07,654::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:07,693::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:07,693::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-141::DEBUG::2014-10-15 06:47:07,694::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:07,695::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:07,696::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-142::DEBUG::2014-10-15 06:47:07,697::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-142::DEBUG::2014-10-15 06:47:07,697::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:08,603::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:09,604::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:10,605::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:10,819::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:10,820::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-143::DEBUG::2014-10-15 06:47:10,821::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:10,822::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:10,823::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-144::DEBUG::2014-10-15 06:47:10,824::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-144::DEBUG::2014-10-15 06:47:10,824::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:11,606::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:12,608::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:12,660::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:12,661::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:12,661::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:47:13,609::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:13,845::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:13,845::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-145::DEBUG::2014-10-15 06:47:13,846::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:13,847::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:13,848::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-146::DEBUG::2014-10-15 06:47:13,849::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-146::DEBUG::2014-10-15 06:47:13,849::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:14,610::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:15,611::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:47:15,829::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:15,830::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:15,830::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:15,830::__init__::232::IOProcess::(_processLogs) (114) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:47:15,830::__init__::232::IOProcess::(_processLogs) (114) Queuing response >Thread-25::DEBUG::2014-10-15 06:47:15,830::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:47:15,836::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000425137 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:47:15,837::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:15,837::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:15,837::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:15,837::__init__::232::IOProcess::(_processLogs) (115) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:47:15,838::__init__::232::IOProcess::(_processLogs) (115) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:15,838::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:15,838::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:15,838::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:15,838::__init__::232::IOProcess::(_processLogs) (116) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:15,839::__init__::232::IOProcess::(_processLogs) (116) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:15,839::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:15,839::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:15,839::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:15,839::__init__::232::IOProcess::(_processLogs) (117) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:15,839::__init__::232::IOProcess::(_processLogs) (117) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:15,840::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:15,840::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:15,840::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:15,840::__init__::232::IOProcess::(_processLogs) (118) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:15,840::__init__::232::IOProcess::(_processLogs) (118) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:47:16,613::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:16,877::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:16,877::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-147::DEBUG::2014-10-15 06:47:16,879::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc8b9b90-1c80-4073-a4cb-db875907ae96`::moving from state init -> state preparing >Thread-147::INFO::2014-10-15 06:47:16,879::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-147::INFO::2014-10-15 06:47:16,879::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425137', 'lastCheck': '1.0', 'valid': True}} >Thread-147::DEBUG::2014-10-15 06:47:16,879::task::1191::Storage.TaskManager.Task::(prepare) Task=`bc8b9b90-1c80-4073-a4cb-db875907ae96`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425137', 'lastCheck': '1.0', 'valid': True}} >Thread-147::DEBUG::2014-10-15 06:47:16,879::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc8b9b90-1c80-4073-a4cb-db875907ae96`::moving from state preparing -> state finished >Thread-147::DEBUG::2014-10-15 06:47:16,879::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-147::DEBUG::2014-10-15 06:47:16,879::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-147::DEBUG::2014-10-15 06:47:16,880::task::993::Storage.TaskManager.Task::(_decref) Task=`bc8b9b90-1c80-4073-a4cb-db875907ae96`::ref 0 aborting False >Thread-147::ERROR::2014-10-15 06:47:16,880::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-147::DEBUG::2014-10-15 06:47:16,881::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:16,893::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:16,894::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-148::DEBUG::2014-10-15 06:47:16,895::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-148::DEBUG::2014-10-15 06:47:16,895::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:17,614::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:17,667::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:17,667::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:17,668::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:47:18,615::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:19,616::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:20,004::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:20,005::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-149::DEBUG::2014-10-15 06:47:20,006::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:20,007::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:20,008::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-150::DEBUG::2014-10-15 06:47:20,009::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-150::DEBUG::2014-10-15 06:47:20,009::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:20,618::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:21,619::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:22,620::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:22,673::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:22,674::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:22,674::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:23,041::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:23,042::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-151::DEBUG::2014-10-15 06:47:23,043::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:23,044::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:23,044::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-152::DEBUG::2014-10-15 06:47:23,045::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-152::DEBUG::2014-10-15 06:47:23,045::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:23,622::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:24,623::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:25,625::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:47:25,841::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:25,841::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:25,841::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:25,841::__init__::232::IOProcess::(_processLogs) (119) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:47:25,842::__init__::232::IOProcess::(_processLogs) (119) Queuing response >Thread-25::DEBUG::2014-10-15 06:47:25,842::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:47:25,848::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00040189 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:47:25,849::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:25,849::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:25,849::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:25,849::__init__::232::IOProcess::(_processLogs) (120) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:47:25,850::__init__::232::IOProcess::(_processLogs) (120) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:25,850::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:25,850::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:25,850::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:25,850::__init__::232::IOProcess::(_processLogs) (121) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:25,851::__init__::232::IOProcess::(_processLogs) (121) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:25,851::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:25,851::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:25,851::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:25,851::__init__::232::IOProcess::(_processLogs) (122) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:25,851::__init__::232::IOProcess::(_processLogs) (122) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:25,852::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:25,852::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:25,852::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:25,852::__init__::232::IOProcess::(_processLogs) (123) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:25,852::__init__::232::IOProcess::(_processLogs) (123) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:26,066::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:26,067::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-153::DEBUG::2014-10-15 06:47:26,068::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:26,069::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:26,069::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-154::DEBUG::2014-10-15 06:47:26,070::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-154::DEBUG::2014-10-15 06:47:26,071::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:26,626::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:27,627::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:27,678::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:27,678::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:27,679::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:47:28,629::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:29,090::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:29,091::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-155::DEBUG::2014-10-15 06:47:29,092::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:29,093::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:29,093::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-156::DEBUG::2014-10-15 06:47:29,094::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-156::DEBUG::2014-10-15 06:47:29,095::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:29,630::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:30,631::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:31,633::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:32,116::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:32,117::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-157::DEBUG::2014-10-15 06:47:32,120::task::595::Storage.TaskManager.Task::(_updateState) Task=`38a25d38-798f-4f76-b69c-05d40b70181e`::moving from state init -> state preparing >Thread-157::INFO::2014-10-15 06:47:32,121::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-157::INFO::2014-10-15 06:47:32,121::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00040189', 'lastCheck': '6.3', 'valid': True}} >Thread-157::DEBUG::2014-10-15 06:47:32,122::task::1191::Storage.TaskManager.Task::(prepare) Task=`38a25d38-798f-4f76-b69c-05d40b70181e`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00040189', 'lastCheck': '6.3', 'valid': True}} >Thread-157::DEBUG::2014-10-15 06:47:32,122::task::595::Storage.TaskManager.Task::(_updateState) Task=`38a25d38-798f-4f76-b69c-05d40b70181e`::moving from state preparing -> state finished >Thread-157::DEBUG::2014-10-15 06:47:32,123::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-157::DEBUG::2014-10-15 06:47:32,123::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-157::DEBUG::2014-10-15 06:47:32,124::task::993::Storage.TaskManager.Task::(_decref) Task=`38a25d38-798f-4f76-b69c-05d40b70181e`::ref 0 aborting False >Thread-157::ERROR::2014-10-15 06:47:32,126::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-157::DEBUG::2014-10-15 06:47:32,128::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:32,145::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:32,146::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-158::DEBUG::2014-10-15 06:47:32,149::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-158::DEBUG::2014-10-15 06:47:32,150::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:32,634::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:32,684::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:32,685::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:32,686::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:47:33,636::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:34,637::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:35,262::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:35,262::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-159::DEBUG::2014-10-15 06:47:35,263::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:35,265::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:35,265::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-160::DEBUG::2014-10-15 06:47:35,266::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-160::DEBUG::2014-10-15 06:47:35,266::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:35,638::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:47:35,853::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:35,853::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:35,853::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:35,853::__init__::232::IOProcess::(_processLogs) (124) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:47:35,854::__init__::232::IOProcess::(_processLogs) (124) Queuing response >Thread-25::DEBUG::2014-10-15 06:47:35,854::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:47:35,860::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000429331 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:47:35,861::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:35,861::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:35,861::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:35,861::__init__::232::IOProcess::(_processLogs) (125) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:47:35,861::__init__::232::IOProcess::(_processLogs) (125) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:35,862::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:35,862::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:35,862::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:35,862::__init__::232::IOProcess::(_processLogs) (126) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:35,862::__init__::232::IOProcess::(_processLogs) (126) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:35,862::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:35,863::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:35,863::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:35,863::__init__::232::IOProcess::(_processLogs) (127) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:35,863::__init__::232::IOProcess::(_processLogs) (127) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:35,863::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:35,863::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:35,863::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:35,864::__init__::232::IOProcess::(_processLogs) (128) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:35,864::__init__::232::IOProcess::(_processLogs) (128) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:47:36,639::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:37,641::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:37,641::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:47:37,641::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:37,692::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:37,692::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:37,693::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:38,315::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:38,316::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-161::DEBUG::2014-10-15 06:47:38,317::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:38,318::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:38,318::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-162::DEBUG::2014-10-15 06:47:38,319::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-162::DEBUG::2014-10-15 06:47:38,319::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:38,642::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:39,644::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:40,645::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:41,336::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:41,337::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-163::DEBUG::2014-10-15 06:47:41,338::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:41,339::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:41,340::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-164::DEBUG::2014-10-15 06:47:41,341::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-164::DEBUG::2014-10-15 06:47:41,341::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:41,646::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:42,647::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:42,719::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:42,719::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:42,720::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:47:43,649::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:44,361::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:44,361::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-165::DEBUG::2014-10-15 06:47:44,362::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:44,364::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:44,364::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-166::DEBUG::2014-10-15 06:47:44,365::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-166::DEBUG::2014-10-15 06:47:44,365::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:44,649::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:45,651::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:47:45,865::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:45,865::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:45,865::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:45,865::__init__::232::IOProcess::(_processLogs) (129) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:47:45,865::__init__::232::IOProcess::(_processLogs) (129) Queuing response >Thread-25::DEBUG::2014-10-15 06:47:45,865::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:47:45,871::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000359356 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:47:45,872::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:45,872::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:45,872::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:45,872::__init__::232::IOProcess::(_processLogs) (130) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:47:45,872::__init__::232::IOProcess::(_processLogs) (130) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:45,873::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:45,873::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:45,873::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:45,873::__init__::232::IOProcess::(_processLogs) (131) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:45,873::__init__::232::IOProcess::(_processLogs) (131) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:45,874::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:45,874::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:45,874::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:45,874::__init__::232::IOProcess::(_processLogs) (132) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:45,874::__init__::232::IOProcess::(_processLogs) (132) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:45,875::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:45,875::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:45,875::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:45,875::__init__::232::IOProcess::(_processLogs) (133) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:45,875::__init__::232::IOProcess::(_processLogs) (133) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:47:46,652::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:47,385::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:47,386::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-167::DEBUG::2014-10-15 06:47:47,387::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f7e9b7f-6060-46d4-bd1c-dcf20da91ae4`::moving from state init -> state preparing >Thread-167::INFO::2014-10-15 06:47:47,387::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-167::INFO::2014-10-15 06:47:47,387::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000359356', 'lastCheck': '1.5', 'valid': True}} >Thread-167::DEBUG::2014-10-15 06:47:47,388::task::1191::Storage.TaskManager.Task::(prepare) Task=`2f7e9b7f-6060-46d4-bd1c-dcf20da91ae4`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000359356', 'lastCheck': '1.5', 'valid': True}} >Thread-167::DEBUG::2014-10-15 06:47:47,388::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f7e9b7f-6060-46d4-bd1c-dcf20da91ae4`::moving from state preparing -> state finished >Thread-167::DEBUG::2014-10-15 06:47:47,388::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-167::DEBUG::2014-10-15 06:47:47,388::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-167::DEBUG::2014-10-15 06:47:47,388::task::993::Storage.TaskManager.Task::(_decref) Task=`2f7e9b7f-6060-46d4-bd1c-dcf20da91ae4`::ref 0 aborting False >Thread-167::ERROR::2014-10-15 06:47:47,389::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-167::DEBUG::2014-10-15 06:47:47,389::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:47,398::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:47,398::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-168::DEBUG::2014-10-15 06:47:47,399::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-168::DEBUG::2014-10-15 06:47:47,399::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:47,653::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:47,725::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:47,726::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:47,726::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:47:48,654::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:49,655::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:50,513::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:50,513::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-169::DEBUG::2014-10-15 06:47:50,514::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:50,516::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:50,516::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-170::DEBUG::2014-10-15 06:47:50,517::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-170::DEBUG::2014-10-15 06:47:50,517::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:50,656::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:51,657::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:52,658::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:52,732::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:52,733::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:52,733::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:53,550::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:53,551::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-171::DEBUG::2014-10-15 06:47:53,552::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:53,553::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:53,554::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-172::DEBUG::2014-10-15 06:47:53,555::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-172::DEBUG::2014-10-15 06:47:53,555::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:53,660::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:54,661::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:55,662::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:47:55,876::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:55,876::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:55,876::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:55,876::__init__::232::IOProcess::(_processLogs) (134) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:47:55,876::__init__::232::IOProcess::(_processLogs) (134) Queuing response >Thread-25::DEBUG::2014-10-15 06:47:55,877::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:47:55,883::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000367053 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:47:55,883::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:55,883::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:55,884::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:55,884::__init__::232::IOProcess::(_processLogs) (135) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:47:55,884::__init__::232::IOProcess::(_processLogs) (135) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:55,884::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:55,885::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:55,885::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:55,885::__init__::232::IOProcess::(_processLogs) (136) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:55,885::__init__::232::IOProcess::(_processLogs) (136) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:55,885::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:55,885::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:55,885::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:55,885::__init__::232::IOProcess::(_processLogs) (137) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:55,886::__init__::232::IOProcess::(_processLogs) (137) Queuing response >Thread-16::DEBUG::2014-10-15 06:47:55,886::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:47:55,886::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:47:55,886::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:47:55,886::__init__::232::IOProcess::(_processLogs) (138) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:47:55,886::__init__::232::IOProcess::(_processLogs) (138) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:56,579::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:56,579::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-173::DEBUG::2014-10-15 06:47:56,581::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:56,582::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:56,582::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-174::DEBUG::2014-10-15 06:47:56,583::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-174::DEBUG::2014-10-15 06:47:56,584::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:56,663::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:47:57,664::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:57,739::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:57,739::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:47:57,740::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:47:58,665::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:59,604::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:59,605::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-175::DEBUG::2014-10-15 06:47:59,606::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:47:59,608::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:47:59,608::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-176::DEBUG::2014-10-15 06:47:59,609::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-176::DEBUG::2014-10-15 06:47:59,609::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:47:59,666::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:00,667::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:01,669::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:02,626::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:02,626::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-177::DEBUG::2014-10-15 06:48:02,627::task::595::Storage.TaskManager.Task::(_updateState) Task=`7fdd6590-c3b1-459a-a5e7-770144ea31aa`::moving from state init -> state preparing >Thread-177::INFO::2014-10-15 06:48:02,628::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-177::INFO::2014-10-15 06:48:02,628::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000367053', 'lastCheck': '6.7', 'valid': True}} >Thread-177::DEBUG::2014-10-15 06:48:02,628::task::1191::Storage.TaskManager.Task::(prepare) Task=`7fdd6590-c3b1-459a-a5e7-770144ea31aa`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000367053', 'lastCheck': '6.7', 'valid': True}} >Thread-177::DEBUG::2014-10-15 06:48:02,628::task::595::Storage.TaskManager.Task::(_updateState) Task=`7fdd6590-c3b1-459a-a5e7-770144ea31aa`::moving from state preparing -> state finished >Thread-177::DEBUG::2014-10-15 06:48:02,628::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-177::DEBUG::2014-10-15 06:48:02,628::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-177::DEBUG::2014-10-15 06:48:02,628::task::993::Storage.TaskManager.Task::(_decref) Task=`7fdd6590-c3b1-459a-a5e7-770144ea31aa`::ref 0 aborting False >Thread-177::ERROR::2014-10-15 06:48:02,629::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-177::DEBUG::2014-10-15 06:48:02,630::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:02,639::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:02,639::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-178::DEBUG::2014-10-15 06:48:02,640::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-178::DEBUG::2014-10-15 06:48:02,641::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:02,669::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:02,745::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:02,746::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:02,746::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 06:48:03,126::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e56fecf-a87e-4a3b-9960-0644f2f72463`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 06:48:03,126::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:48:03,127::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:03,127::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:03,127::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:03,127::__init__::232::IOProcess::(_processLogs) (139) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:48:03,128::__init__::232::IOProcess::(_processLogs) (139) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:03,129::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:03,129::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:03,129::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:03,129::__init__::232::IOProcess::(_processLogs) (140) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:48:03,129::__init__::232::IOProcess::(_processLogs) (140) Queuing response >Thread-17::INFO::2014-10-15 06:48:03,129::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:48:03,130::task::1191::Storage.TaskManager.Task::(prepare) Task=`6e56fecf-a87e-4a3b-9960-0644f2f72463`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:48:03,130::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e56fecf-a87e-4a3b-9960-0644f2f72463`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 06:48:03,130::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 06:48:03,130::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 06:48:03,130::task::993::Storage.TaskManager.Task::(_decref) Task=`6e56fecf-a87e-4a3b-9960-0644f2f72463`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:48:03,671::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:04,672::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:05,673::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:05,754::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:05,754::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-179::DEBUG::2014-10-15 06:48:05,755::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:05,757::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:05,757::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-180::DEBUG::2014-10-15 06:48:05,758::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-180::DEBUG::2014-10-15 06:48:05,758::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 06:48:05,887::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:05,887::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:05,887::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:05,888::__init__::232::IOProcess::(_processLogs) (141) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:48:05,888::__init__::232::IOProcess::(_processLogs) (141) Queuing response >Thread-25::DEBUG::2014-10-15 06:48:05,888::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:48:05,895::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000803496 s, 571 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:48:05,896::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:05,896::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:05,896::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:05,896::__init__::232::IOProcess::(_processLogs) (142) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:48:05,896::__init__::232::IOProcess::(_processLogs) (142) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:05,897::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:05,897::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:05,897::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:05,897::__init__::232::IOProcess::(_processLogs) (143) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:05,897::__init__::232::IOProcess::(_processLogs) (143) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:05,898::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:05,898::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:05,898::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:05,898::__init__::232::IOProcess::(_processLogs) (144) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:05,898::__init__::232::IOProcess::(_processLogs) (144) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:05,898::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:05,899::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:05,899::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:05,899::__init__::232::IOProcess::(_processLogs) (145) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:05,899::__init__::232::IOProcess::(_processLogs) (145) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:48:06,674::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:07,675::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:07,675::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:48:07,675::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:07,752::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:07,752::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:07,753::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:48:08,676::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:08,808::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:08,808::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-181::DEBUG::2014-10-15 06:48:08,809::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:08,811::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:08,811::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-182::DEBUG::2014-10-15 06:48:08,812::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-182::DEBUG::2014-10-15 06:48:08,812::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:09,678::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:10,679::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:11,680::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:11,827::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:11,827::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-183::DEBUG::2014-10-15 06:48:11,828::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:11,829::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:11,830::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-184::DEBUG::2014-10-15 06:48:11,830::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-184::DEBUG::2014-10-15 06:48:11,831::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:12,681::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:12,758::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:12,759::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:12,759::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:48:13,682::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:14,683::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:14,849::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:14,850::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-185::DEBUG::2014-10-15 06:48:14,851::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:14,852::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:14,852::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-186::DEBUG::2014-10-15 06:48:14,853::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-186::DEBUG::2014-10-15 06:48:14,854::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:15,684::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:48:15,900::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:15,900::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:15,900::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:15,900::__init__::232::IOProcess::(_processLogs) (146) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:48:15,900::__init__::232::IOProcess::(_processLogs) (146) Queuing response >Thread-25::DEBUG::2014-10-15 06:48:15,901::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:48:15,907::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000385678 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:48:15,907::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:15,908::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:15,908::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:15,908::__init__::232::IOProcess::(_processLogs) (147) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:48:15,908::__init__::232::IOProcess::(_processLogs) (147) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:15,908::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:15,909::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:15,909::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:15,909::__init__::232::IOProcess::(_processLogs) (148) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:15,909::__init__::232::IOProcess::(_processLogs) (148) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:15,909::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:15,909::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:15,909::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:15,910::__init__::232::IOProcess::(_processLogs) (149) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:15,910::__init__::232::IOProcess::(_processLogs) (149) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:15,910::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:15,910::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:15,910::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:15,910::__init__::232::IOProcess::(_processLogs) (150) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:15,910::__init__::232::IOProcess::(_processLogs) (150) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:48:16,686::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:17,687::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:17,764::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:17,764::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:17,765::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:17,872::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:17,873::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-187::DEBUG::2014-10-15 06:48:17,874::task::595::Storage.TaskManager.Task::(_updateState) Task=`71d6fa66-49a8-445a-bcdf-e596ec48cfe1`::moving from state init -> state preparing >Thread-187::INFO::2014-10-15 06:48:17,875::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-187::INFO::2014-10-15 06:48:17,875::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000385678', 'lastCheck': '2.0', 'valid': True}} >Thread-187::DEBUG::2014-10-15 06:48:17,875::task::1191::Storage.TaskManager.Task::(prepare) Task=`71d6fa66-49a8-445a-bcdf-e596ec48cfe1`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000385678', 'lastCheck': '2.0', 'valid': True}} >Thread-187::DEBUG::2014-10-15 06:48:17,875::task::595::Storage.TaskManager.Task::(_updateState) Task=`71d6fa66-49a8-445a-bcdf-e596ec48cfe1`::moving from state preparing -> state finished >Thread-187::DEBUG::2014-10-15 06:48:17,875::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-187::DEBUG::2014-10-15 06:48:17,875::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-187::DEBUG::2014-10-15 06:48:17,875::task::993::Storage.TaskManager.Task::(_decref) Task=`71d6fa66-49a8-445a-bcdf-e596ec48cfe1`::ref 0 aborting False >Thread-187::ERROR::2014-10-15 06:48:17,876::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-187::DEBUG::2014-10-15 06:48:17,877::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:17,888::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:17,888::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-188::DEBUG::2014-10-15 06:48:17,889::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-188::DEBUG::2014-10-15 06:48:17,890::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:18,688::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:19,689::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:20,690::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:21,061::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:21,062::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-189::DEBUG::2014-10-15 06:48:21,063::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:21,064::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:21,065::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-190::DEBUG::2014-10-15 06:48:21,065::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-190::DEBUG::2014-10-15 06:48:21,066::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:21,691::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:22,692::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:22,770::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:22,771::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:22,771::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:48:23,694::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:24,096::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:24,097::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-191::DEBUG::2014-10-15 06:48:24,098::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:24,100::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:24,100::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-192::DEBUG::2014-10-15 06:48:24,101::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-192::DEBUG::2014-10-15 06:48:24,101::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:24,695::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:25,696::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:48:25,911::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:25,911::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:25,912::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:25,912::__init__::232::IOProcess::(_processLogs) (151) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:48:25,912::__init__::232::IOProcess::(_processLogs) (151) Queuing response >Thread-25::DEBUG::2014-10-15 06:48:25,912::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:48:25,918::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000410981 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:48:25,919::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:25,919::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:25,919::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:25,919::__init__::232::IOProcess::(_processLogs) (152) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:48:25,919::__init__::232::IOProcess::(_processLogs) (152) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:25,920::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:25,920::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:25,920::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:25,920::__init__::232::IOProcess::(_processLogs) (153) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:25,921::__init__::232::IOProcess::(_processLogs) (153) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:25,921::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:25,921::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:25,921::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:25,921::__init__::232::IOProcess::(_processLogs) (154) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:25,921::__init__::232::IOProcess::(_processLogs) (154) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:25,922::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:25,922::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:25,922::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:25,922::__init__::232::IOProcess::(_processLogs) (155) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:25,922::__init__::232::IOProcess::(_processLogs) (155) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:48:26,697::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:27,124::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:27,124::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-193::DEBUG::2014-10-15 06:48:27,126::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:27,127::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:27,127::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-194::DEBUG::2014-10-15 06:48:27,128::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-194::DEBUG::2014-10-15 06:48:27,128::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:27,698::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:27,777::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:27,777::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:27,778::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:48:28,700::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:29,701::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:30,164::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:30,164::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-195::DEBUG::2014-10-15 06:48:30,166::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:30,167::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:30,167::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-196::DEBUG::2014-10-15 06:48:30,168::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-196::DEBUG::2014-10-15 06:48:30,169::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:30,702::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:31,703::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:32,705::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:32,783::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:32,784::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:32,784::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:33,190::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:33,191::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-197::DEBUG::2014-10-15 06:48:33,192::task::595::Storage.TaskManager.Task::(_updateState) Task=`10fb55d9-439d-4868-a9b4-ea529e7740de`::moving from state init -> state preparing >Thread-197::INFO::2014-10-15 06:48:33,192::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-197::INFO::2014-10-15 06:48:33,192::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000410981', 'lastCheck': '7.3', 'valid': True}} >Thread-197::DEBUG::2014-10-15 06:48:33,192::task::1191::Storage.TaskManager.Task::(prepare) Task=`10fb55d9-439d-4868-a9b4-ea529e7740de`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000410981', 'lastCheck': '7.3', 'valid': True}} >Thread-197::DEBUG::2014-10-15 06:48:33,193::task::595::Storage.TaskManager.Task::(_updateState) Task=`10fb55d9-439d-4868-a9b4-ea529e7740de`::moving from state preparing -> state finished >Thread-197::DEBUG::2014-10-15 06:48:33,193::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-197::DEBUG::2014-10-15 06:48:33,193::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-197::DEBUG::2014-10-15 06:48:33,193::task::993::Storage.TaskManager.Task::(_decref) Task=`10fb55d9-439d-4868-a9b4-ea529e7740de`::ref 0 aborting False >Thread-197::ERROR::2014-10-15 06:48:33,194::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-197::DEBUG::2014-10-15 06:48:33,194::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:33,206::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:33,207::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-198::DEBUG::2014-10-15 06:48:33,208::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-198::DEBUG::2014-10-15 06:48:33,208::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:33,706::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:34,707::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:35,709::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:48:35,923::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:35,923::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:35,923::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:35,923::__init__::232::IOProcess::(_processLogs) (156) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:48:35,924::__init__::232::IOProcess::(_processLogs) (156) Queuing response >Thread-25::DEBUG::2014-10-15 06:48:35,924::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:48:35,930::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000423823 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:48:35,931::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:35,931::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:35,931::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:35,931::__init__::232::IOProcess::(_processLogs) (157) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:48:35,931::__init__::232::IOProcess::(_processLogs) (157) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:35,932::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:35,932::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:35,932::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:35,932::__init__::232::IOProcess::(_processLogs) (158) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:35,932::__init__::232::IOProcess::(_processLogs) (158) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:35,933::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:35,933::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:35,933::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:35,933::__init__::232::IOProcess::(_processLogs) (159) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:35,933::__init__::232::IOProcess::(_processLogs) (159) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:35,933::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:35,934::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:35,934::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:35,934::__init__::232::IOProcess::(_processLogs) (160) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:35,934::__init__::232::IOProcess::(_processLogs) (160) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:36,388::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:36,389::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-199::DEBUG::2014-10-15 06:48:36,390::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:36,391::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:36,392::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-200::DEBUG::2014-10-15 06:48:36,393::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-200::DEBUG::2014-10-15 06:48:36,393::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:36,710::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:37,711::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:37,711::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:48:37,711::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:37,790::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:37,790::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:37,791::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:48:38,713::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:39,436::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:39,437::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-201::DEBUG::2014-10-15 06:48:39,438::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:39,439::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:39,440::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-202::DEBUG::2014-10-15 06:48:39,441::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-202::DEBUG::2014-10-15 06:48:39,441::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:39,714::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:40,715::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:41,716::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:42,459::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:42,460::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-203::DEBUG::2014-10-15 06:48:42,460::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:42,462::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:42,462::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-204::DEBUG::2014-10-15 06:48:42,463::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-204::DEBUG::2014-10-15 06:48:42,463::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:42,718::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:42,796::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:42,797::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:42,798::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:48:43,719::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:44,720::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:45,488::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:45,489::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-205::DEBUG::2014-10-15 06:48:45,490::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:45,491::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:45,491::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-206::DEBUG::2014-10-15 06:48:45,492::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-206::DEBUG::2014-10-15 06:48:45,492::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:45,721::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:48:45,935::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:45,935::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:45,935::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:45,935::__init__::232::IOProcess::(_processLogs) (161) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:48:45,935::__init__::232::IOProcess::(_processLogs) (161) Queuing response >Thread-25::DEBUG::2014-10-15 06:48:45,936::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:48:45,942::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000436928 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:48:45,942::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:45,943::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:45,943::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:45,943::__init__::232::IOProcess::(_processLogs) (162) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:48:45,943::__init__::232::IOProcess::(_processLogs) (162) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:45,943::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:45,944::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:45,944::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:45,944::__init__::232::IOProcess::(_processLogs) (163) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:45,944::__init__::232::IOProcess::(_processLogs) (163) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:45,944::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:45,944::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:45,944::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:45,944::__init__::232::IOProcess::(_processLogs) (164) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:45,945::__init__::232::IOProcess::(_processLogs) (164) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:45,945::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:45,945::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:45,945::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:45,945::__init__::232::IOProcess::(_processLogs) (165) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:45,945::__init__::232::IOProcess::(_processLogs) (165) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:48:46,723::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:47,724::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:47,800::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:47,800::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:47,801::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:48,513::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:48,513::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-207::DEBUG::2014-10-15 06:48:48,515::task::595::Storage.TaskManager.Task::(_updateState) Task=`21608aab-fe5a-4d41-8a33-48c5769b04d6`::moving from state init -> state preparing >Thread-207::INFO::2014-10-15 06:48:48,515::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-207::INFO::2014-10-15 06:48:48,515::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000436928', 'lastCheck': '2.6', 'valid': True}} >Thread-207::DEBUG::2014-10-15 06:48:48,515::task::1191::Storage.TaskManager.Task::(prepare) Task=`21608aab-fe5a-4d41-8a33-48c5769b04d6`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000436928', 'lastCheck': '2.6', 'valid': True}} >Thread-207::DEBUG::2014-10-15 06:48:48,515::task::595::Storage.TaskManager.Task::(_updateState) Task=`21608aab-fe5a-4d41-8a33-48c5769b04d6`::moving from state preparing -> state finished >Thread-207::DEBUG::2014-10-15 06:48:48,515::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-207::DEBUG::2014-10-15 06:48:48,515::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-207::DEBUG::2014-10-15 06:48:48,516::task::993::Storage.TaskManager.Task::(_decref) Task=`21608aab-fe5a-4d41-8a33-48c5769b04d6`::ref 0 aborting False >Thread-207::ERROR::2014-10-15 06:48:48,516::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-207::DEBUG::2014-10-15 06:48:48,517::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:48,529::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:48,529::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-208::DEBUG::2014-10-15 06:48:48,530::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-208::DEBUG::2014-10-15 06:48:48,531::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:48,725::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:49,726::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:50,727::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:51,657::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:51,658::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-209::DEBUG::2014-10-15 06:48:51,659::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:51,660::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:51,661::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-210::DEBUG::2014-10-15 06:48:51,662::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-210::DEBUG::2014-10-15 06:48:51,662::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:51,728::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:52,729::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:52,806::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:52,807::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:52,807::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:48:53,731::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:54,695::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:54,696::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-211::DEBUG::2014-10-15 06:48:54,697::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:54,698::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:54,699::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-212::DEBUG::2014-10-15 06:48:54,700::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-212::DEBUG::2014-10-15 06:48:54,700::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:54,732::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:55,733::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:48:55,946::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:55,946::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:55,947::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:55,947::__init__::232::IOProcess::(_processLogs) (166) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:48:55,947::__init__::232::IOProcess::(_processLogs) (166) Queuing response >Thread-25::DEBUG::2014-10-15 06:48:55,947::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:48:55,953::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000444433 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:48:55,954::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:55,954::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:55,954::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:55,954::__init__::232::IOProcess::(_processLogs) (167) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:48:55,954::__init__::232::IOProcess::(_processLogs) (167) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:55,955::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:55,955::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:55,955::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:55,955::__init__::232::IOProcess::(_processLogs) (168) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:55,955::__init__::232::IOProcess::(_processLogs) (168) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:55,956::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:55,956::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:55,956::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:55,956::__init__::232::IOProcess::(_processLogs) (169) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:55,956::__init__::232::IOProcess::(_processLogs) (169) Queuing response >Thread-16::DEBUG::2014-10-15 06:48:55,956::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:48:55,957::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:48:55,957::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:48:55,957::__init__::232::IOProcess::(_processLogs) (170) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:48:55,957::__init__::232::IOProcess::(_processLogs) (170) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:48:56,734::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:57,728::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:57,728::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-213::DEBUG::2014-10-15 06:48:57,730::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:48:57,731::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:48:57,731::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-214::DEBUG::2014-10-15 06:48:57,732::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-214::DEBUG::2014-10-15 06:48:57,733::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:48:57,735::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:57,813::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:57,813::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:48:57,814::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:48:58,737::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:48:59,738::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:00,739::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:00,750::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:00,750::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-215::DEBUG::2014-10-15 06:49:00,752::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:00,753::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:00,753::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-216::DEBUG::2014-10-15 06:49:00,754::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-216::DEBUG::2014-10-15 06:49:00,754::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:01,740::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:02,741::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:02,815::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:02,815::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:02,816::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 06:49:03,169::task::595::Storage.TaskManager.Task::(_updateState) Task=`79c64cef-7881-48ac-83ff-3f9b74f738bb`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 06:49:03,169::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:49:03,169::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:03,169::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:03,170::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:03,170::__init__::232::IOProcess::(_processLogs) (171) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:49:03,171::__init__::232::IOProcess::(_processLogs) (171) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:03,171::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:03,171::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:03,172::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:03,172::__init__::232::IOProcess::(_processLogs) (172) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:49:03,172::__init__::232::IOProcess::(_processLogs) (172) Queuing response >Thread-17::INFO::2014-10-15 06:49:03,172::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:49:03,172::task::1191::Storage.TaskManager.Task::(prepare) Task=`79c64cef-7881-48ac-83ff-3f9b74f738bb`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:49:03,172::task::595::Storage.TaskManager.Task::(_updateState) Task=`79c64cef-7881-48ac-83ff-3f9b74f738bb`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 06:49:03,172::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 06:49:03,172::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 06:49:03,173::task::993::Storage.TaskManager.Task::(_decref) Task=`79c64cef-7881-48ac-83ff-3f9b74f738bb`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:49:03,742::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:03,770::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:03,771::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-217::DEBUG::2014-10-15 06:49:03,772::task::595::Storage.TaskManager.Task::(_updateState) Task=`a708fec7-0db4-4971-9362-b7e180c38157`::moving from state init -> state preparing >Thread-217::INFO::2014-10-15 06:49:03,772::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-217::INFO::2014-10-15 06:49:03,772::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000444433', 'lastCheck': '7.8', 'valid': True}} >Thread-217::DEBUG::2014-10-15 06:49:03,772::task::1191::Storage.TaskManager.Task::(prepare) Task=`a708fec7-0db4-4971-9362-b7e180c38157`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000444433', 'lastCheck': '7.8', 'valid': True}} >Thread-217::DEBUG::2014-10-15 06:49:03,772::task::595::Storage.TaskManager.Task::(_updateState) Task=`a708fec7-0db4-4971-9362-b7e180c38157`::moving from state preparing -> state finished >Thread-217::DEBUG::2014-10-15 06:49:03,773::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-217::DEBUG::2014-10-15 06:49:03,773::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-217::DEBUG::2014-10-15 06:49:03,773::task::993::Storage.TaskManager.Task::(_decref) Task=`a708fec7-0db4-4971-9362-b7e180c38157`::ref 0 aborting False >Thread-217::ERROR::2014-10-15 06:49:03,774::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-217::DEBUG::2014-10-15 06:49:03,774::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:03,783::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:03,784::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-218::DEBUG::2014-10-15 06:49:03,785::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-218::DEBUG::2014-10-15 06:49:03,785::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:04,744::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:05,745::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:49:05,958::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:05,958::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:05,958::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:05,958::__init__::232::IOProcess::(_processLogs) (173) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:49:05,958::__init__::232::IOProcess::(_processLogs) (173) Queuing response >Thread-25::DEBUG::2014-10-15 06:49:05,959::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:49:05,965::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000387446 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:49:05,966::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:05,966::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:05,966::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:05,966::__init__::232::IOProcess::(_processLogs) (174) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:49:05,966::__init__::232::IOProcess::(_processLogs) (174) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:05,967::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:05,967::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:05,967::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:05,967::__init__::232::IOProcess::(_processLogs) (175) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:05,967::__init__::232::IOProcess::(_processLogs) (175) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:05,968::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:05,968::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:05,968::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:05,968::__init__::232::IOProcess::(_processLogs) (176) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:05,968::__init__::232::IOProcess::(_processLogs) (176) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:05,968::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:05,968::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:05,969::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:05,969::__init__::232::IOProcess::(_processLogs) (177) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:05,969::__init__::232::IOProcess::(_processLogs) (177) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:49:06,746::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:06,880::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:06,880::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-219::DEBUG::2014-10-15 06:49:06,882::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:06,882::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:06,883::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-220::DEBUG::2014-10-15 06:49:06,884::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-220::DEBUG::2014-10-15 06:49:06,884::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:07,747::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:07,747::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:49:07,747::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:07,821::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:07,822::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:07,822::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:49:08,749::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:09,750::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:09,921::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:09,922::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-221::DEBUG::2014-10-15 06:49:09,923::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:09,924::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:09,924::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-222::DEBUG::2014-10-15 06:49:09,925::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-222::DEBUG::2014-10-15 06:49:09,926::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:10,751::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:11,753::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:12,754::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:12,828::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:12,828::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:12,829::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:12,942::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:12,942::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-223::DEBUG::2014-10-15 06:49:12,943::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:12,944::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:12,945::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-224::DEBUG::2014-10-15 06:49:12,946::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-224::DEBUG::2014-10-15 06:49:12,946::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:13,755::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:14,756::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:15,757::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:15,959::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:15,959::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-225::DEBUG::2014-10-15 06:49:15,960::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:15,962::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:15,962::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-226::DEBUG::2014-10-15 06:49:15,963::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-226::DEBUG::2014-10-15 06:49:15,963::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 06:49:15,970::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:15,970::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:15,970::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:15,970::__init__::232::IOProcess::(_processLogs) (178) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:49:15,970::__init__::232::IOProcess::(_processLogs) (178) Queuing response >Thread-25::DEBUG::2014-10-15 06:49:15,971::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:49:15,977::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000384825 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:49:15,977::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:15,977::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:15,977::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:15,978::__init__::232::IOProcess::(_processLogs) (179) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:49:15,978::__init__::232::IOProcess::(_processLogs) (179) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:15,978::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:15,978::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:15,979::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:15,979::__init__::232::IOProcess::(_processLogs) (180) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:15,979::__init__::232::IOProcess::(_processLogs) (180) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:15,979::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:15,979::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:15,979::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:15,979::__init__::232::IOProcess::(_processLogs) (181) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:15,980::__init__::232::IOProcess::(_processLogs) (181) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:15,980::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:15,980::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:15,980::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:15,980::__init__::232::IOProcess::(_processLogs) (182) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:15,980::__init__::232::IOProcess::(_processLogs) (182) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:49:16,759::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:17,760::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:17,835::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:17,835::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:17,835::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:49:18,761::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:18,986::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:18,987::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-227::DEBUG::2014-10-15 06:49:18,988::task::595::Storage.TaskManager.Task::(_updateState) Task=`96c1d509-fe32-4ce3-bb05-4afef56ee12f`::moving from state init -> state preparing >Thread-227::INFO::2014-10-15 06:49:18,988::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-227::INFO::2014-10-15 06:49:18,989::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000384825', 'lastCheck': '3.0', 'valid': True}} >Thread-227::DEBUG::2014-10-15 06:49:18,989::task::1191::Storage.TaskManager.Task::(prepare) Task=`96c1d509-fe32-4ce3-bb05-4afef56ee12f`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000384825', 'lastCheck': '3.0', 'valid': True}} >Thread-227::DEBUG::2014-10-15 06:49:18,989::task::595::Storage.TaskManager.Task::(_updateState) Task=`96c1d509-fe32-4ce3-bb05-4afef56ee12f`::moving from state preparing -> state finished >Thread-227::DEBUG::2014-10-15 06:49:18,989::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-227::DEBUG::2014-10-15 06:49:18,989::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-227::DEBUG::2014-10-15 06:49:18,989::task::993::Storage.TaskManager.Task::(_decref) Task=`96c1d509-fe32-4ce3-bb05-4afef56ee12f`::ref 0 aborting False >Thread-227::ERROR::2014-10-15 06:49:18,990::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-227::DEBUG::2014-10-15 06:49:18,991::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:19,001::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:19,002::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-228::DEBUG::2014-10-15 06:49:19,003::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-228::DEBUG::2014-10-15 06:49:19,003::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:19,762::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:20,764::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:21,765::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:22,101::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:22,102::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-229::DEBUG::2014-10-15 06:49:22,103::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:22,104::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:22,105::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-230::DEBUG::2014-10-15 06:49:22,106::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-230::DEBUG::2014-10-15 06:49:22,106::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:22,766::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:22,841::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:22,842::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:22,842::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:49:23,767::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:24,769::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:25,138::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:25,139::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-231::DEBUG::2014-10-15 06:49:25,140::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:25,141::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:25,141::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-232::DEBUG::2014-10-15 06:49:25,142::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-232::DEBUG::2014-10-15 06:49:25,143::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:25,770::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-25::DEBUG::2014-10-15 06:49:25,981::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 06:49:25,982::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:25,982::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:25,982::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:25,982::__init__::232::IOProcess::(_processLogs) (183) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:49:25,982::__init__::232::IOProcess::(_processLogs) (183) Queuing response >Thread-25::DEBUG::2014-10-15 06:49:25,989::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 06:49:25,989::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:25,990::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:25,990::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:25,990::__init__::232::IOProcess::(_processLogs) (184) Got request for method 'touch' >Thread-16::DEBUG::2014-10-15 06:49:26,025::__init__::232::IOProcess::(_processLogs) (184) Queuing response >Thread-25::DEBUG::2014-10-15 06:49:26,025::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-16::DEBUG::2014-10-15 06:49:26,026::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:26,026::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:26,026::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:26,026::__init__::232::IOProcess::(_processLogs) (185) Got request for method 'readfile' >Thread-16::DEBUG::2014-10-15 06:49:26,026::__init__::232::IOProcess::(_processLogs) (185) Queuing response >Thread-25::DEBUG::2014-10-15 06:49:26,027::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=domain1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-0000000003e1', 'REMOTE_PATH=10.66.128.104:/home/NFS1', 'ROLE=Master', 'SDUUID=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b90f33781d11f6e6b0b3945de3167c0719a458c1'] >Thread-16::DEBUG::2014-10-15 06:49:26,027::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:26,027::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:26,027::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:26,027::__init__::232::IOProcess::(_processLogs) (186) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:26,028::__init__::232::IOProcess::(_processLogs) (186) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:26,028::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:26,028::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:26,028::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:26,028::__init__::232::IOProcess::(_processLogs) (187) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:49:26,028::__init__::232::IOProcess::(_processLogs) (187) Queuing response >Thread-25::DEBUG::2014-10-15 06:49:26,028::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-25::INFO::2014-10-15 06:49:26,029::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_imageNS already registered >Thread-25::INFO::2014-10-15 06:49:26,029::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_volumeNS already registered >Thread-16::DEBUG::2014-10-15 06:49:26,029::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:26,029::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:26,029::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:26,029::__init__::232::IOProcess::(_processLogs) (188) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:49:26,029::__init__::232::IOProcess::(_processLogs) (188) Queuing response >Thread-25::DEBUG::2014-10-15 06:49:26,030::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:49:26,036::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000355495 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:49:26,036::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:26,036::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:26,036::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:26,037::__init__::232::IOProcess::(_processLogs) (189) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:49:26,037::__init__::232::IOProcess::(_processLogs) (189) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:26,037::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:26,038::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:26,038::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:26,038::__init__::232::IOProcess::(_processLogs) (190) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:26,038::__init__::232::IOProcess::(_processLogs) (190) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:26,038::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:26,038::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:26,038::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:26,038::__init__::232::IOProcess::(_processLogs) (191) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:26,039::__init__::232::IOProcess::(_processLogs) (191) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:26,039::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:26,039::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:26,039::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:26,039::__init__::232::IOProcess::(_processLogs) (192) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:26,039::__init__::232::IOProcess::(_processLogs) (192) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:49:26,771::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:27,771::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:27,848::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:27,848::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:27,849::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:28,161::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:28,162::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-234::DEBUG::2014-10-15 06:49:28,163::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:28,164::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:28,164::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-235::DEBUG::2014-10-15 06:49:28,165::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-235::DEBUG::2014-10-15 06:49:28,166::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:28,772::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:29,773::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:30,774::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:31,188::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:31,188::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-236::DEBUG::2014-10-15 06:49:31,190::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:31,191::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:31,192::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-237::DEBUG::2014-10-15 06:49:31,193::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-237::DEBUG::2014-10-15 06:49:31,193::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:31,776::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:32,777::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:32,855::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:32,855::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:32,855::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:49:33,778::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:34,216::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:34,217::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-238::DEBUG::2014-10-15 06:49:34,218::task::595::Storage.TaskManager.Task::(_updateState) Task=`73db151d-5550-4284-8bbb-36e82eae112a`::moving from state init -> state preparing >Thread-238::INFO::2014-10-15 06:49:34,218::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-238::INFO::2014-10-15 06:49:34,219::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000355495', 'lastCheck': '8.2', 'valid': True}} >Thread-238::DEBUG::2014-10-15 06:49:34,219::task::1191::Storage.TaskManager.Task::(prepare) Task=`73db151d-5550-4284-8bbb-36e82eae112a`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000355495', 'lastCheck': '8.2', 'valid': True}} >Thread-238::DEBUG::2014-10-15 06:49:34,219::task::595::Storage.TaskManager.Task::(_updateState) Task=`73db151d-5550-4284-8bbb-36e82eae112a`::moving from state preparing -> state finished >Thread-238::DEBUG::2014-10-15 06:49:34,219::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-238::DEBUG::2014-10-15 06:49:34,219::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-238::DEBUG::2014-10-15 06:49:34,219::task::993::Storage.TaskManager.Task::(_decref) Task=`73db151d-5550-4284-8bbb-36e82eae112a`::ref 0 aborting False >Thread-238::ERROR::2014-10-15 06:49:34,220::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-238::DEBUG::2014-10-15 06:49:34,221::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:34,232::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:34,232::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-239::DEBUG::2014-10-15 06:49:34,233::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-239::DEBUG::2014-10-15 06:49:34,234::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:34,780::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:35,781::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:49:36,040::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:36,040::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:36,040::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:36,041::__init__::232::IOProcess::(_processLogs) (193) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:49:36,041::__init__::232::IOProcess::(_processLogs) (193) Queuing response >Thread-25::DEBUG::2014-10-15 06:49:36,041::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:49:36,047::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000433746 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:49:36,048::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:36,048::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:36,048::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:36,048::__init__::232::IOProcess::(_processLogs) (194) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:49:36,048::__init__::232::IOProcess::(_processLogs) (194) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:36,049::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:36,049::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:36,049::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:36,049::__init__::232::IOProcess::(_processLogs) (195) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:36,049::__init__::232::IOProcess::(_processLogs) (195) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:36,050::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:36,050::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:36,050::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:36,050::__init__::232::IOProcess::(_processLogs) (196) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:36,050::__init__::232::IOProcess::(_processLogs) (196) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:36,050::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:36,050::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:36,051::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:36,051::__init__::232::IOProcess::(_processLogs) (197) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:36,051::__init__::232::IOProcess::(_processLogs) (197) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:49:36,782::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:37,353::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:37,353::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-240::DEBUG::2014-10-15 06:49:37,355::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:37,355::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:37,356::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-241::DEBUG::2014-10-15 06:49:37,357::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-241::DEBUG::2014-10-15 06:49:37,358::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:37,783::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:37,783::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:49:37,784::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:37,861::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:37,862::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:37,862::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:49:38,785::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:39,786::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:40,383::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:40,384::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-242::DEBUG::2014-10-15 06:49:40,385::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:40,386::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:40,387::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-243::DEBUG::2014-10-15 06:49:40,388::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-243::DEBUG::2014-10-15 06:49:40,388::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:40,788::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:41,789::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:42,790::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:42,868::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:42,868::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:42,869::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:43,431::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:43,431::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-244::DEBUG::2014-10-15 06:49:43,432::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:43,433::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:43,434::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-245::DEBUG::2014-10-15 06:49:43,435::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-245::DEBUG::2014-10-15 06:49:43,435::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:43,791::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:44,792::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:45,792::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:49:46,052::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:46,052::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:46,052::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:46,052::__init__::232::IOProcess::(_processLogs) (198) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:49:46,052::__init__::232::IOProcess::(_processLogs) (198) Queuing response >Thread-25::DEBUG::2014-10-15 06:49:46,053::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:49:46,059::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000377751 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:49:46,059::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:46,059::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:46,060::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:46,060::__init__::232::IOProcess::(_processLogs) (199) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:49:46,060::__init__::232::IOProcess::(_processLogs) (199) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:46,060::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:46,061::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:46,061::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:46,061::__init__::232::IOProcess::(_processLogs) (200) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:46,061::__init__::232::IOProcess::(_processLogs) (200) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:46,061::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:46,061::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:46,061::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:46,062::__init__::232::IOProcess::(_processLogs) (201) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:46,062::__init__::232::IOProcess::(_processLogs) (201) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:46,062::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:46,062::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:46,062::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:46,062::__init__::232::IOProcess::(_processLogs) (202) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:46,062::__init__::232::IOProcess::(_processLogs) (202) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:46,457::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:46,457::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-246::DEBUG::2014-10-15 06:49:46,458::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:46,459::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:46,460::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-247::DEBUG::2014-10-15 06:49:46,461::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-247::DEBUG::2014-10-15 06:49:46,461::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:46,794::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:47,795::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:47,873::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:47,874::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:47,874::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:49:48,796::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:49,485::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:49,486::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-248::DEBUG::2014-10-15 06:49:49,488::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ab5920c-0fac-43cd-a497-5595be345027`::moving from state init -> state preparing >Thread-248::INFO::2014-10-15 06:49:49,488::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-248::INFO::2014-10-15 06:49:49,488::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000377751', 'lastCheck': '3.4', 'valid': True}} >Thread-248::DEBUG::2014-10-15 06:49:49,488::task::1191::Storage.TaskManager.Task::(prepare) Task=`4ab5920c-0fac-43cd-a497-5595be345027`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000377751', 'lastCheck': '3.4', 'valid': True}} >Thread-248::DEBUG::2014-10-15 06:49:49,488::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ab5920c-0fac-43cd-a497-5595be345027`::moving from state preparing -> state finished >Thread-248::DEBUG::2014-10-15 06:49:49,488::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-248::DEBUG::2014-10-15 06:49:49,488::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-248::DEBUG::2014-10-15 06:49:49,489::task::993::Storage.TaskManager.Task::(_decref) Task=`4ab5920c-0fac-43cd-a497-5595be345027`::ref 0 aborting False >Thread-248::ERROR::2014-10-15 06:49:49,489::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-248::DEBUG::2014-10-15 06:49:49,490::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:49,502::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:49,503::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-249::DEBUG::2014-10-15 06:49:49,504::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-249::DEBUG::2014-10-15 06:49:49,504::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:49,797::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:50,799::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:51,800::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:52,604::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:52,605::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-250::DEBUG::2014-10-15 06:49:52,606::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:52,607::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:52,608::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-251::DEBUG::2014-10-15 06:49:52,608::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-251::DEBUG::2014-10-15 06:49:52,609::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:52,801::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:52,876::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:52,876::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:52,877::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:49:53,802::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:54,804::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:55,644::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:55,644::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-252::DEBUG::2014-10-15 06:49:55,645::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:55,646::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:55,647::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-253::DEBUG::2014-10-15 06:49:55,648::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-253::DEBUG::2014-10-15 06:49:55,648::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:55,805::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:49:56,063::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:56,063::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:56,064::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:56,064::__init__::232::IOProcess::(_processLogs) (203) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:49:56,064::__init__::232::IOProcess::(_processLogs) (203) Queuing response >Thread-25::DEBUG::2014-10-15 06:49:56,064::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:49:56,070::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000367199 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:49:56,071::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:56,071::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:56,071::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:56,071::__init__::232::IOProcess::(_processLogs) (204) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:49:56,071::__init__::232::IOProcess::(_processLogs) (204) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:56,072::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:56,072::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:56,072::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:56,072::__init__::232::IOProcess::(_processLogs) (205) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:56,072::__init__::232::IOProcess::(_processLogs) (205) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:56,073::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:56,073::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:56,073::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:56,073::__init__::232::IOProcess::(_processLogs) (206) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:56,073::__init__::232::IOProcess::(_processLogs) (206) Queuing response >Thread-16::DEBUG::2014-10-15 06:49:56,073::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:49:56,073::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:49:56,073::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:49:56,074::__init__::232::IOProcess::(_processLogs) (207) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:49:56,074::__init__::232::IOProcess::(_processLogs) (207) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:49:56,806::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:57,807::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:57,878::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:57,878::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:49:57,879::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:58,663::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:58,664::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-254::DEBUG::2014-10-15 06:49:58,665::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:49:58,666::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:49:58,666::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-255::DEBUG::2014-10-15 06:49:58,667::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-255::DEBUG::2014-10-15 06:49:58,667::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:49:58,808::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:49:59,809::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:00,810::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:01,693::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:01,695::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-256::DEBUG::2014-10-15 06:50:01,698::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:01,700::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:01,702::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-257::DEBUG::2014-10-15 06:50:01,704::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-257::DEBUG::2014-10-15 06:50:01,706::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:01,812::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:02,813::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:02,884::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:02,885::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:02,885::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 06:50:03,211::task::595::Storage.TaskManager.Task::(_updateState) Task=`68746404-b713-469f-abe9-8cbf867e4110`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 06:50:03,211::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:50:03,212::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:03,212::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:03,212::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:03,212::__init__::232::IOProcess::(_processLogs) (208) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:50:03,213::__init__::232::IOProcess::(_processLogs) (208) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:03,214::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:03,214::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:03,214::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:03,214::__init__::232::IOProcess::(_processLogs) (209) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:50:03,214::__init__::232::IOProcess::(_processLogs) (209) Queuing response >Thread-17::INFO::2014-10-15 06:50:03,215::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:50:03,215::task::1191::Storage.TaskManager.Task::(prepare) Task=`68746404-b713-469f-abe9-8cbf867e4110`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:50:03,215::task::595::Storage.TaskManager.Task::(_updateState) Task=`68746404-b713-469f-abe9-8cbf867e4110`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 06:50:03,215::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 06:50:03,215::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 06:50:03,215::task::993::Storage.TaskManager.Task::(_decref) Task=`68746404-b713-469f-abe9-8cbf867e4110`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:50:03,814::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:04,724::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:04,725::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-258::DEBUG::2014-10-15 06:50:04,726::task::595::Storage.TaskManager.Task::(_updateState) Task=`71ba544c-50c2-404a-9458-419c197b6b30`::moving from state init -> state preparing >Thread-258::INFO::2014-10-15 06:50:04,726::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-258::INFO::2014-10-15 06:50:04,727::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000367199', 'lastCheck': '8.7', 'valid': True}} >Thread-258::DEBUG::2014-10-15 06:50:04,727::task::1191::Storage.TaskManager.Task::(prepare) Task=`71ba544c-50c2-404a-9458-419c197b6b30`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000367199', 'lastCheck': '8.7', 'valid': True}} >Thread-258::DEBUG::2014-10-15 06:50:04,727::task::595::Storage.TaskManager.Task::(_updateState) Task=`71ba544c-50c2-404a-9458-419c197b6b30`::moving from state preparing -> state finished >Thread-258::DEBUG::2014-10-15 06:50:04,727::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-258::DEBUG::2014-10-15 06:50:04,727::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-258::DEBUG::2014-10-15 06:50:04,727::task::993::Storage.TaskManager.Task::(_decref) Task=`71ba544c-50c2-404a-9458-419c197b6b30`::ref 0 aborting False >Thread-258::ERROR::2014-10-15 06:50:04,728::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-258::DEBUG::2014-10-15 06:50:04,729::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:04,741::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:04,742::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-259::DEBUG::2014-10-15 06:50:04,743::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-259::DEBUG::2014-10-15 06:50:04,743::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:04,815::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:05,817::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:50:06,075::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:06,075::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:06,075::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:06,075::__init__::232::IOProcess::(_processLogs) (210) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:50:06,075::__init__::232::IOProcess::(_processLogs) (210) Queuing response >Thread-25::DEBUG::2014-10-15 06:50:06,075::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:50:06,082::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000358008 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:50:06,083::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:06,083::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:06,083::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:06,083::__init__::232::IOProcess::(_processLogs) (211) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:50:06,083::__init__::232::IOProcess::(_processLogs) (211) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:06,084::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:06,084::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:06,084::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:06,084::__init__::232::IOProcess::(_processLogs) (212) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:06,084::__init__::232::IOProcess::(_processLogs) (212) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:06,085::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:06,085::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:06,085::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:06,085::__init__::232::IOProcess::(_processLogs) (213) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:06,085::__init__::232::IOProcess::(_processLogs) (213) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:06,085::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:06,085::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:06,086::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:06,086::__init__::232::IOProcess::(_processLogs) (214) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:06,086::__init__::232::IOProcess::(_processLogs) (214) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:50:06,818::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:07,819::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:07,819::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:50:07,819::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:07,845::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:07,846::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-260::DEBUG::2014-10-15 06:50:07,846::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:07,848::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:07,848::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-261::DEBUG::2014-10-15 06:50:07,849::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-261::DEBUG::2014-10-15 06:50:07,849::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:07,891::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:07,892::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:07,892::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:50:08,821::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:09,822::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:10,823::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:10,888::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:10,889::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-262::DEBUG::2014-10-15 06:50:10,890::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:10,891::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:10,892::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-263::DEBUG::2014-10-15 06:50:10,893::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-263::DEBUG::2014-10-15 06:50:10,893::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:11,825::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:12,826::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:12,898::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:12,898::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:12,899::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:50:13,827::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:13,917::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:13,917::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-264::DEBUG::2014-10-15 06:50:13,918::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:13,919::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:13,920::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-265::DEBUG::2014-10-15 06:50:13,921::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-265::DEBUG::2014-10-15 06:50:13,921::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:14,828::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:15,830::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:50:16,087::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:16,087::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:16,087::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:16,087::__init__::232::IOProcess::(_processLogs) (215) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:50:16,087::__init__::232::IOProcess::(_processLogs) (215) Queuing response >Thread-25::DEBUG::2014-10-15 06:50:16,088::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:50:16,094::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000420462 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:50:16,095::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:16,095::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:16,095::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:16,095::__init__::232::IOProcess::(_processLogs) (216) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:50:16,095::__init__::232::IOProcess::(_processLogs) (216) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:16,096::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:16,096::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:16,096::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:16,096::__init__::232::IOProcess::(_processLogs) (217) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:16,096::__init__::232::IOProcess::(_processLogs) (217) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:16,096::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:16,097::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:16,097::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:16,097::__init__::232::IOProcess::(_processLogs) (218) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:16,097::__init__::232::IOProcess::(_processLogs) (218) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:16,097::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:16,097::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:16,097::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:16,097::__init__::232::IOProcess::(_processLogs) (219) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:16,098::__init__::232::IOProcess::(_processLogs) (219) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:50:16,831::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:16,943::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:16,944::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-266::DEBUG::2014-10-15 06:50:16,945::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:16,946::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:16,947::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-267::DEBUG::2014-10-15 06:50:16,948::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-267::DEBUG::2014-10-15 06:50:16,948::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:17,832::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:17,904::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:17,905::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:17,905::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:50:18,833::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:19,835::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:19,971::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:19,971::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-268::DEBUG::2014-10-15 06:50:19,973::task::595::Storage.TaskManager.Task::(_updateState) Task=`36253797-a8b7-4fd4-a853-fd2850f6f467`::moving from state init -> state preparing >Thread-268::INFO::2014-10-15 06:50:19,973::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-268::INFO::2014-10-15 06:50:19,973::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420462', 'lastCheck': '3.9', 'valid': True}} >Thread-268::DEBUG::2014-10-15 06:50:19,973::task::1191::Storage.TaskManager.Task::(prepare) Task=`36253797-a8b7-4fd4-a853-fd2850f6f467`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420462', 'lastCheck': '3.9', 'valid': True}} >Thread-268::DEBUG::2014-10-15 06:50:19,973::task::595::Storage.TaskManager.Task::(_updateState) Task=`36253797-a8b7-4fd4-a853-fd2850f6f467`::moving from state preparing -> state finished >Thread-268::DEBUG::2014-10-15 06:50:19,974::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-268::DEBUG::2014-10-15 06:50:19,974::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-268::DEBUG::2014-10-15 06:50:19,974::task::993::Storage.TaskManager.Task::(_decref) Task=`36253797-a8b7-4fd4-a853-fd2850f6f467`::ref 0 aborting False >Thread-268::ERROR::2014-10-15 06:50:19,975::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-268::DEBUG::2014-10-15 06:50:19,975::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:19,986::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:19,986::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-269::DEBUG::2014-10-15 06:50:19,987::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-269::DEBUG::2014-10-15 06:50:19,988::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:20,836::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:21,837::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:22,838::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:22,908::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:22,908::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:22,909::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:23,086::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:23,087::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-270::DEBUG::2014-10-15 06:50:23,088::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:23,089::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:23,090::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-271::DEBUG::2014-10-15 06:50:23,090::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-271::DEBUG::2014-10-15 06:50:23,091::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:23,840::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:24,840::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:25,842::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:50:26,098::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:26,099::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:26,099::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:26,099::__init__::232::IOProcess::(_processLogs) (220) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:50:26,099::__init__::232::IOProcess::(_processLogs) (220) Queuing response >Thread-25::DEBUG::2014-10-15 06:50:26,099::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:50:26,105::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000419285 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:50:26,106::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:26,106::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:26,106::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:26,106::__init__::232::IOProcess::(_processLogs) (221) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:50:26,106::__init__::232::IOProcess::(_processLogs) (221) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:26,107::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:26,107::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:26,107::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:26,107::__init__::232::IOProcess::(_processLogs) (222) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:26,107::__init__::232::IOProcess::(_processLogs) (222) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:26,107::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:26,107::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:26,108::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:26,108::__init__::232::IOProcess::(_processLogs) (223) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:26,108::__init__::232::IOProcess::(_processLogs) (223) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:26,108::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:26,108::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:26,108::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:26,108::__init__::232::IOProcess::(_processLogs) (224) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:26,109::__init__::232::IOProcess::(_processLogs) (224) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:26,115::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:26,115::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-272::DEBUG::2014-10-15 06:50:26,116::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:26,117::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:26,118::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-273::DEBUG::2014-10-15 06:50:26,119::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-273::DEBUG::2014-10-15 06:50:26,119::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:26,843::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:27,844::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:27,911::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:27,911::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:27,912::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:50:28,845::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:29,168::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:29,169::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-274::DEBUG::2014-10-15 06:50:29,170::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:29,171::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:29,172::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-275::DEBUG::2014-10-15 06:50:29,172::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-275::DEBUG::2014-10-15 06:50:29,173::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:29,847::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:30,848::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:31,849::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:32,195::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:32,196::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-276::DEBUG::2014-10-15 06:50:32,199::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:32,201::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:32,202::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-277::DEBUG::2014-10-15 06:50:32,204::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-277::DEBUG::2014-10-15 06:50:32,205::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:32,850::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:32,918::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:32,918::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:32,918::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:50:33,851::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:34,853::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:35,221::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:35,221::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-278::DEBUG::2014-10-15 06:50:35,223::task::595::Storage.TaskManager.Task::(_updateState) Task=`040006d6-e2e4-4636-81eb-85f79b63193a`::moving from state init -> state preparing >Thread-278::INFO::2014-10-15 06:50:35,223::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-278::INFO::2014-10-15 06:50:35,223::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419285', 'lastCheck': '9.1', 'valid': True}} >Thread-278::DEBUG::2014-10-15 06:50:35,223::task::1191::Storage.TaskManager.Task::(prepare) Task=`040006d6-e2e4-4636-81eb-85f79b63193a`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419285', 'lastCheck': '9.1', 'valid': True}} >Thread-278::DEBUG::2014-10-15 06:50:35,223::task::595::Storage.TaskManager.Task::(_updateState) Task=`040006d6-e2e4-4636-81eb-85f79b63193a`::moving from state preparing -> state finished >Thread-278::DEBUG::2014-10-15 06:50:35,223::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-278::DEBUG::2014-10-15 06:50:35,223::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-278::DEBUG::2014-10-15 06:50:35,224::task::993::Storage.TaskManager.Task::(_decref) Task=`040006d6-e2e4-4636-81eb-85f79b63193a`::ref 0 aborting False >Thread-278::ERROR::2014-10-15 06:50:35,224::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-278::DEBUG::2014-10-15 06:50:35,225::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:35,232::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:35,232::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-279::DEBUG::2014-10-15 06:50:35,233::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-279::DEBUG::2014-10-15 06:50:35,234::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:35,854::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:50:36,110::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:36,110::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:36,110::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:36,110::__init__::232::IOProcess::(_processLogs) (225) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:50:36,110::__init__::232::IOProcess::(_processLogs) (225) Queuing response >Thread-25::DEBUG::2014-10-15 06:50:36,110::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:50:36,116::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000430856 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:50:36,117::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:36,117::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:36,117::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:36,118::__init__::232::IOProcess::(_processLogs) (226) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:50:36,118::__init__::232::IOProcess::(_processLogs) (226) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:36,118::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:36,118::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:36,118::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:36,119::__init__::232::IOProcess::(_processLogs) (227) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:36,119::__init__::232::IOProcess::(_processLogs) (227) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:36,119::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:36,119::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:36,119::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:36,119::__init__::232::IOProcess::(_processLogs) (228) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:36,120::__init__::232::IOProcess::(_processLogs) (228) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:36,120::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:36,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:36,120::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:36,120::__init__::232::IOProcess::(_processLogs) (229) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:36,120::__init__::232::IOProcess::(_processLogs) (229) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:50:36,855::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:37,856::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:37,857::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:50:37,857::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:37,924::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:37,925::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:37,925::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:38,350::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:38,351::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-280::DEBUG::2014-10-15 06:50:38,352::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:38,353::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:38,353::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-281::DEBUG::2014-10-15 06:50:38,354::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-281::DEBUG::2014-10-15 06:50:38,354::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:38,858::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:39,859::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:40,861::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:41,385::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:41,386::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-282::DEBUG::2014-10-15 06:50:41,387::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:41,388::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:41,389::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-283::DEBUG::2014-10-15 06:50:41,390::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-283::DEBUG::2014-10-15 06:50:41,390::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:41,862::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:42,863::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:42,931::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:42,932::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:42,932::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:50:43,864::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:44,414::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:44,414::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-284::DEBUG::2014-10-15 06:50:44,415::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:44,416::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:44,417::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-285::DEBUG::2014-10-15 06:50:44,418::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-285::DEBUG::2014-10-15 06:50:44,418::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:44,866::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:45,867::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:50:46,121::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:46,121::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:46,122::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:46,122::__init__::232::IOProcess::(_processLogs) (230) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:50:46,122::__init__::232::IOProcess::(_processLogs) (230) Queuing response >Thread-25::DEBUG::2014-10-15 06:50:46,122::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:50:46,128::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000358345 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:50:46,129::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:46,129::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:46,129::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:46,129::__init__::232::IOProcess::(_processLogs) (231) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:50:46,129::__init__::232::IOProcess::(_processLogs) (231) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:46,130::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:46,130::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:46,130::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:46,130::__init__::232::IOProcess::(_processLogs) (232) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:46,130::__init__::232::IOProcess::(_processLogs) (232) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:46,131::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:46,131::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:46,131::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:46,131::__init__::232::IOProcess::(_processLogs) (233) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:46,131::__init__::232::IOProcess::(_processLogs) (233) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:46,131::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:46,132::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:46,132::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:46,132::__init__::232::IOProcess::(_processLogs) (234) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:46,132::__init__::232::IOProcess::(_processLogs) (234) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:50:46,868::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:47,447::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:47,447::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-286::DEBUG::2014-10-15 06:50:47,448::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:47,449::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:47,450::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-287::DEBUG::2014-10-15 06:50:47,451::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-287::DEBUG::2014-10-15 06:50:47,451::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:47,869::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:47,938::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:47,938::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:47,939::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:50:48,871::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:49,872::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:50,473::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:50,473::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-288::DEBUG::2014-10-15 06:50:50,475::task::595::Storage.TaskManager.Task::(_updateState) Task=`9cdc3a3a-81d1-4605-8265-6c2930f39611`::moving from state init -> state preparing >Thread-288::INFO::2014-10-15 06:50:50,475::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-288::INFO::2014-10-15 06:50:50,475::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000358345', 'lastCheck': '4.3', 'valid': True}} >Thread-288::DEBUG::2014-10-15 06:50:50,475::task::1191::Storage.TaskManager.Task::(prepare) Task=`9cdc3a3a-81d1-4605-8265-6c2930f39611`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000358345', 'lastCheck': '4.3', 'valid': True}} >Thread-288::DEBUG::2014-10-15 06:50:50,475::task::595::Storage.TaskManager.Task::(_updateState) Task=`9cdc3a3a-81d1-4605-8265-6c2930f39611`::moving from state preparing -> state finished >Thread-288::DEBUG::2014-10-15 06:50:50,475::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-288::DEBUG::2014-10-15 06:50:50,475::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-288::DEBUG::2014-10-15 06:50:50,476::task::993::Storage.TaskManager.Task::(_decref) Task=`9cdc3a3a-81d1-4605-8265-6c2930f39611`::ref 0 aborting False >Thread-288::ERROR::2014-10-15 06:50:50,476::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-288::DEBUG::2014-10-15 06:50:50,477::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:50,488::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:50,488::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-289::DEBUG::2014-10-15 06:50:50,489::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-289::DEBUG::2014-10-15 06:50:50,490::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:50,873::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:51,874::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:52,876::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:52,945::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:52,945::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:52,946::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:53,661::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:53,662::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-290::DEBUG::2014-10-15 06:50:53,663::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:53,664::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:53,664::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-291::DEBUG::2014-10-15 06:50:53,665::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-291::DEBUG::2014-10-15 06:50:53,666::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:53,877::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:54,878::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:55,879::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:50:56,133::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:56,133::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:56,133::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:56,133::__init__::232::IOProcess::(_processLogs) (235) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:50:56,133::__init__::232::IOProcess::(_processLogs) (235) Queuing response >Thread-25::DEBUG::2014-10-15 06:50:56,134::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:50:56,140::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00039181 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:50:56,140::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:56,141::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:56,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:56,141::__init__::232::IOProcess::(_processLogs) (236) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:50:56,141::__init__::232::IOProcess::(_processLogs) (236) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:56,141::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:56,142::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:56,142::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:56,142::__init__::232::IOProcess::(_processLogs) (237) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:56,142::__init__::232::IOProcess::(_processLogs) (237) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:56,142::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:56,142::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:56,142::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:56,142::__init__::232::IOProcess::(_processLogs) (238) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:56,143::__init__::232::IOProcess::(_processLogs) (238) Queuing response >Thread-16::DEBUG::2014-10-15 06:50:56,143::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:50:56,143::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:50:56,143::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:50:56,143::__init__::232::IOProcess::(_processLogs) (239) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:50:56,143::__init__::232::IOProcess::(_processLogs) (239) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:56,697::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:56,698::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-292::DEBUG::2014-10-15 06:50:56,699::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:56,700::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:56,700::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-293::DEBUG::2014-10-15 06:50:56,701::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-293::DEBUG::2014-10-15 06:50:56,702::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:56,881::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:50:57,882::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:57,948::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:57,948::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:50:57,949::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:50:58,883::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:59,720::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:59,721::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-294::DEBUG::2014-10-15 06:50:59,722::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:50:59,723::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:50:59,723::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-295::DEBUG::2014-10-15 06:50:59,724::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-295::DEBUG::2014-10-15 06:50:59,725::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:50:59,884::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:00,885::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:01,887::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:02,748::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:02,748::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-296::DEBUG::2014-10-15 06:51:02,749::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:02,751::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:02,751::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-297::DEBUG::2014-10-15 06:51:02,752::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-297::DEBUG::2014-10-15 06:51:02,753::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:02,888::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:02,954::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:02,954::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:02,955::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 06:51:03,252::task::595::Storage.TaskManager.Task::(_updateState) Task=`10d8ff8d-8d4c-46a7-ad2f-6c4c0d733c4e`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 06:51:03,252::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:51:03,253::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:03,253::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:03,253::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:03,253::__init__::232::IOProcess::(_processLogs) (240) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:51:03,254::__init__::232::IOProcess::(_processLogs) (240) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:03,255::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:03,255::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:03,255::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:03,255::__init__::232::IOProcess::(_processLogs) (241) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:51:03,255::__init__::232::IOProcess::(_processLogs) (241) Queuing response >Thread-17::INFO::2014-10-15 06:51:03,256::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:51:03,256::task::1191::Storage.TaskManager.Task::(prepare) Task=`10d8ff8d-8d4c-46a7-ad2f-6c4c0d733c4e`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:51:03,256::task::595::Storage.TaskManager.Task::(_updateState) Task=`10d8ff8d-8d4c-46a7-ad2f-6c4c0d733c4e`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 06:51:03,256::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 06:51:03,256::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 06:51:03,256::task::993::Storage.TaskManager.Task::(_decref) Task=`10d8ff8d-8d4c-46a7-ad2f-6c4c0d733c4e`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:51:03,890::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:04,891::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:05,778::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:05,779::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-298::DEBUG::2014-10-15 06:51:05,780::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7cedea4-1792-40d7-b891-0abfb908cafc`::moving from state init -> state preparing >Thread-298::INFO::2014-10-15 06:51:05,780::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-298::INFO::2014-10-15 06:51:05,780::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00039181', 'lastCheck': '9.6', 'valid': True}} >Thread-298::DEBUG::2014-10-15 06:51:05,781::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7cedea4-1792-40d7-b891-0abfb908cafc`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00039181', 'lastCheck': '9.6', 'valid': True}} >Thread-298::DEBUG::2014-10-15 06:51:05,781::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7cedea4-1792-40d7-b891-0abfb908cafc`::moving from state preparing -> state finished >Thread-298::DEBUG::2014-10-15 06:51:05,781::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-298::DEBUG::2014-10-15 06:51:05,781::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-298::DEBUG::2014-10-15 06:51:05,781::task::993::Storage.TaskManager.Task::(_decref) Task=`c7cedea4-1792-40d7-b891-0abfb908cafc`::ref 0 aborting False >Thread-298::ERROR::2014-10-15 06:51:05,782::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-298::DEBUG::2014-10-15 06:51:05,782::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:05,792::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:05,792::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-299::DEBUG::2014-10-15 06:51:05,793::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-299::DEBUG::2014-10-15 06:51:05,793::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:05,892::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:51:06,144::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:06,144::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:06,144::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:06,145::__init__::232::IOProcess::(_processLogs) (242) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:51:06,145::__init__::232::IOProcess::(_processLogs) (242) Queuing response >Thread-25::DEBUG::2014-10-15 06:51:06,145::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:51:06,152::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000404477 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:51:06,152::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:06,152::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:06,153::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:06,153::__init__::232::IOProcess::(_processLogs) (243) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:51:06,153::__init__::232::IOProcess::(_processLogs) (243) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:06,153::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:06,153::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:06,154::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:06,154::__init__::232::IOProcess::(_processLogs) (244) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:06,154::__init__::232::IOProcess::(_processLogs) (244) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:06,154::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:06,154::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:06,154::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:06,154::__init__::232::IOProcess::(_processLogs) (245) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:06,155::__init__::232::IOProcess::(_processLogs) (245) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:06,155::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:06,155::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:06,155::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:06,155::__init__::232::IOProcess::(_processLogs) (246) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:06,155::__init__::232::IOProcess::(_processLogs) (246) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:51:06,893::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:07,895::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:07,895::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:51:07,895::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:07,961::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:07,961::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:07,961::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:51:08,896::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:08,925::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:08,926::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-300::DEBUG::2014-10-15 06:51:08,927::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:08,928::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:08,929::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-301::DEBUG::2014-10-15 06:51:08,930::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-301::DEBUG::2014-10-15 06:51:08,930::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:09,898::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:10,899::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:11,900::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:11,956::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:11,957::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-302::DEBUG::2014-10-15 06:51:11,958::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:11,959::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:11,960::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-303::DEBUG::2014-10-15 06:51:11,961::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-303::DEBUG::2014-10-15 06:51:11,961::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:12,901::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:12,967::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:12,968::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:12,968::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:51:13,902::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:14,903::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:14,983::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:14,984::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-304::DEBUG::2014-10-15 06:51:14,985::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:14,986::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:14,987::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-305::DEBUG::2014-10-15 06:51:14,988::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-305::DEBUG::2014-10-15 06:51:14,988::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:15,904::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:51:16,156::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:16,156::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:16,156::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:16,157::__init__::232::IOProcess::(_processLogs) (247) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:51:16,157::__init__::232::IOProcess::(_processLogs) (247) Queuing response >Thread-25::DEBUG::2014-10-15 06:51:16,157::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:51:16,163::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000408044 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:51:16,164::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:16,164::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:16,164::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:16,164::__init__::232::IOProcess::(_processLogs) (248) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:51:16,164::__init__::232::IOProcess::(_processLogs) (248) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:16,165::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:16,165::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:16,165::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:16,165::__init__::232::IOProcess::(_processLogs) (249) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:16,165::__init__::232::IOProcess::(_processLogs) (249) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:16,165::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:16,166::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:16,166::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:16,166::__init__::232::IOProcess::(_processLogs) (250) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:16,166::__init__::232::IOProcess::(_processLogs) (250) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:16,166::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:16,166::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:16,166::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:16,167::__init__::232::IOProcess::(_processLogs) (251) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:16,167::__init__::232::IOProcess::(_processLogs) (251) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:51:16,906::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:17,907::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:17,974::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:17,974::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:17,975::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:18,020::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:18,021::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-306::DEBUG::2014-10-15 06:51:18,022::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:18,023::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:18,023::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-307::DEBUG::2014-10-15 06:51:18,024::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-307::DEBUG::2014-10-15 06:51:18,024::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:18,908::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:19,909::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:20,910::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:21,073::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:21,074::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-308::DEBUG::2014-10-15 06:51:21,075::task::595::Storage.TaskManager.Task::(_updateState) Task=`67805eae-495a-4525-8fee-af3861a61641`::moving from state init -> state preparing >Thread-308::INFO::2014-10-15 06:51:21,075::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-308::INFO::2014-10-15 06:51:21,076::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000408044', 'lastCheck': '4.9', 'valid': True}} >Thread-308::DEBUG::2014-10-15 06:51:21,076::task::1191::Storage.TaskManager.Task::(prepare) Task=`67805eae-495a-4525-8fee-af3861a61641`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000408044', 'lastCheck': '4.9', 'valid': True}} >Thread-308::DEBUG::2014-10-15 06:51:21,076::task::595::Storage.TaskManager.Task::(_updateState) Task=`67805eae-495a-4525-8fee-af3861a61641`::moving from state preparing -> state finished >Thread-308::DEBUG::2014-10-15 06:51:21,076::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-308::DEBUG::2014-10-15 06:51:21,076::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-308::DEBUG::2014-10-15 06:51:21,076::task::993::Storage.TaskManager.Task::(_decref) Task=`67805eae-495a-4525-8fee-af3861a61641`::ref 0 aborting False >Thread-308::ERROR::2014-10-15 06:51:21,077::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-308::DEBUG::2014-10-15 06:51:21,078::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:21,084::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:21,085::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-309::DEBUG::2014-10-15 06:51:21,086::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-309::DEBUG::2014-10-15 06:51:21,087::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:21,911::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:22,912::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:22,981::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:22,981::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:22,981::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:51:23,913::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:24,194::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:24,195::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-310::DEBUG::2014-10-15 06:51:24,196::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:24,197::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:24,198::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-311::DEBUG::2014-10-15 06:51:24,198::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-311::DEBUG::2014-10-15 06:51:24,199::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:24,915::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:25,916::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:51:26,168::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:26,168::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:26,168::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:26,168::__init__::232::IOProcess::(_processLogs) (252) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:51:26,168::__init__::232::IOProcess::(_processLogs) (252) Queuing response >Thread-25::DEBUG::2014-10-15 06:51:26,169::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:51:26,175::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00036635 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:51:26,175::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:26,175::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:26,175::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:26,176::__init__::232::IOProcess::(_processLogs) (253) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:51:26,176::__init__::232::IOProcess::(_processLogs) (253) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:26,176::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:26,176::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:26,176::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:26,176::__init__::232::IOProcess::(_processLogs) (254) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:26,177::__init__::232::IOProcess::(_processLogs) (254) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:26,177::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:26,177::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:26,177::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:26,177::__init__::232::IOProcess::(_processLogs) (255) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:26,177::__init__::232::IOProcess::(_processLogs) (255) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:26,178::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:26,178::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:26,178::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:26,178::__init__::232::IOProcess::(_processLogs) (256) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:26,178::__init__::232::IOProcess::(_processLogs) (256) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:51:26,917::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:27,223::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:27,224::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-312::DEBUG::2014-10-15 06:51:27,225::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:27,226::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:27,226::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-313::DEBUG::2014-10-15 06:51:27,227::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-313::DEBUG::2014-10-15 06:51:27,228::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:27,918::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:27,987::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:27,988::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:27,988::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:51:28,919::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:29,921::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:30,245::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:30,246::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-314::DEBUG::2014-10-15 06:51:30,247::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:30,248::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:30,249::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-315::DEBUG::2014-10-15 06:51:30,250::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-315::DEBUG::2014-10-15 06:51:30,250::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:30,922::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:31,923::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:32,925::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:32,994::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:32,994::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:32,995::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:33,268::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:33,269::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-316::DEBUG::2014-10-15 06:51:33,270::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:33,271::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:33,272::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-317::DEBUG::2014-10-15 06:51:33,272::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-317::DEBUG::2014-10-15 06:51:33,273::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:33,926::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:34,927::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:35,929::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:51:36,179::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:36,179::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:36,179::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:36,180::__init__::232::IOProcess::(_processLogs) (257) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:51:36,180::__init__::232::IOProcess::(_processLogs) (257) Queuing response >Thread-25::DEBUG::2014-10-15 06:51:36,180::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:51:36,186::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000449531 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:51:36,187::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:36,187::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:36,187::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:36,187::__init__::232::IOProcess::(_processLogs) (258) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:51:36,187::__init__::232::IOProcess::(_processLogs) (258) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:36,188::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:36,188::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:36,188::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:36,188::__init__::232::IOProcess::(_processLogs) (259) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:36,189::__init__::232::IOProcess::(_processLogs) (259) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:36,189::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:36,189::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:36,189::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:36,189::__init__::232::IOProcess::(_processLogs) (260) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:36,189::__init__::232::IOProcess::(_processLogs) (260) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:36,190::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:36,190::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:36,190::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:36,190::__init__::232::IOProcess::(_processLogs) (261) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:36,190::__init__::232::IOProcess::(_processLogs) (261) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:36,288::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:36,289::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-318::DEBUG::2014-10-15 06:51:36,290::task::595::Storage.TaskManager.Task::(_updateState) Task=`b4043d32-bac9-496e-b9a1-8f302554b298`::moving from state init -> state preparing >Thread-318::INFO::2014-10-15 06:51:36,290::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-318::INFO::2014-10-15 06:51:36,291::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000449531', 'lastCheck': '0.1', 'valid': True}} >Thread-318::DEBUG::2014-10-15 06:51:36,291::task::1191::Storage.TaskManager.Task::(prepare) Task=`b4043d32-bac9-496e-b9a1-8f302554b298`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000449531', 'lastCheck': '0.1', 'valid': True}} >Thread-318::DEBUG::2014-10-15 06:51:36,291::task::595::Storage.TaskManager.Task::(_updateState) Task=`b4043d32-bac9-496e-b9a1-8f302554b298`::moving from state preparing -> state finished >Thread-318::DEBUG::2014-10-15 06:51:36,291::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-318::DEBUG::2014-10-15 06:51:36,291::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-318::DEBUG::2014-10-15 06:51:36,291::task::993::Storage.TaskManager.Task::(_decref) Task=`b4043d32-bac9-496e-b9a1-8f302554b298`::ref 0 aborting False >Thread-318::ERROR::2014-10-15 06:51:36,292::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-318::DEBUG::2014-10-15 06:51:36,293::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:36,300::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:36,300::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-319::DEBUG::2014-10-15 06:51:36,301::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-319::DEBUG::2014-10-15 06:51:36,302::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:36,930::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:37,931::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:37,931::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:51:37,931::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:37,998::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:37,998::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:37,999::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:51:38,933::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:39,398::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:39,399::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-320::DEBUG::2014-10-15 06:51:39,400::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:39,401::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:39,401::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-321::DEBUG::2014-10-15 06:51:39,402::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-321::DEBUG::2014-10-15 06:51:39,402::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:39,934::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:40,935::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:41,936::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:42,453::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:42,454::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-322::DEBUG::2014-10-15 06:51:42,455::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:42,456::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:42,456::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-323::DEBUG::2014-10-15 06:51:42,457::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-323::DEBUG::2014-10-15 06:51:42,457::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:42,938::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:43,000::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:43,000::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:43,001::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:51:43,939::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:44,940::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:45,475::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:45,475::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-324::DEBUG::2014-10-15 06:51:45,476::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:45,478::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:45,478::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-325::DEBUG::2014-10-15 06:51:45,479::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-325::DEBUG::2014-10-15 06:51:45,479::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:45,941::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:51:46,191::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:46,191::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:46,191::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:46,191::__init__::232::IOProcess::(_processLogs) (262) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:51:46,191::__init__::232::IOProcess::(_processLogs) (262) Queuing response >Thread-25::DEBUG::2014-10-15 06:51:46,192::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:51:46,197::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000366 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:51:46,198::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:46,198::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:46,198::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:46,198::__init__::232::IOProcess::(_processLogs) (263) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:51:46,199::__init__::232::IOProcess::(_processLogs) (263) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:46,199::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:46,199::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:46,199::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:46,199::__init__::232::IOProcess::(_processLogs) (264) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:46,199::__init__::232::IOProcess::(_processLogs) (264) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:46,200::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:46,200::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:46,200::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:46,200::__init__::232::IOProcess::(_processLogs) (265) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:46,200::__init__::232::IOProcess::(_processLogs) (265) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:46,201::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:46,201::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:46,201::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:46,201::__init__::232::IOProcess::(_processLogs) (266) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:46,201::__init__::232::IOProcess::(_processLogs) (266) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:51:46,943::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:47,944::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:48,007::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:48,007::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:48,008::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:48,557::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:48,558::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-326::DEBUG::2014-10-15 06:51:48,559::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:48,560::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:48,560::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-327::DEBUG::2014-10-15 06:51:48,561::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-327::DEBUG::2014-10-15 06:51:48,561::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:48,945::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:49,946::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:50,948::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:51,608::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:51,609::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-328::DEBUG::2014-10-15 06:51:51,610::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b309372-87bb-4e89-b750-71c18270a9b3`::moving from state init -> state preparing >Thread-328::INFO::2014-10-15 06:51:51,610::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-328::INFO::2014-10-15 06:51:51,610::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000366', 'lastCheck': '5.4', 'valid': True}} >Thread-328::DEBUG::2014-10-15 06:51:51,610::task::1191::Storage.TaskManager.Task::(prepare) Task=`8b309372-87bb-4e89-b750-71c18270a9b3`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000366', 'lastCheck': '5.4', 'valid': True}} >Thread-328::DEBUG::2014-10-15 06:51:51,610::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b309372-87bb-4e89-b750-71c18270a9b3`::moving from state preparing -> state finished >Thread-328::DEBUG::2014-10-15 06:51:51,611::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-328::DEBUG::2014-10-15 06:51:51,611::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-328::DEBUG::2014-10-15 06:51:51,611::task::993::Storage.TaskManager.Task::(_decref) Task=`8b309372-87bb-4e89-b750-71c18270a9b3`::ref 0 aborting False >Thread-328::ERROR::2014-10-15 06:51:51,612::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-328::DEBUG::2014-10-15 06:51:51,612::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:51,622::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:51,622::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-329::DEBUG::2014-10-15 06:51:51,623::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-329::DEBUG::2014-10-15 06:51:51,624::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:51,949::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:52,950::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:53,013::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:53,014::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:53,014::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:51:53,951::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:54,740::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:54,741::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-330::DEBUG::2014-10-15 06:51:54,742::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:54,743::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:54,743::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-331::DEBUG::2014-10-15 06:51:54,744::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-331::DEBUG::2014-10-15 06:51:54,744::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:54,952::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:55,954::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:51:56,202::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:56,202::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:56,202::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:56,202::__init__::232::IOProcess::(_processLogs) (267) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:51:56,203::__init__::232::IOProcess::(_processLogs) (267) Queuing response >Thread-25::DEBUG::2014-10-15 06:51:56,203::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:51:56,209::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00038967 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:51:56,209::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:56,210::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:56,210::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:56,210::__init__::232::IOProcess::(_processLogs) (268) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:51:56,210::__init__::232::IOProcess::(_processLogs) (268) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:56,210::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:56,211::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:56,211::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:56,211::__init__::232::IOProcess::(_processLogs) (269) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:56,211::__init__::232::IOProcess::(_processLogs) (269) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:56,211::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:56,211::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:56,211::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:56,211::__init__::232::IOProcess::(_processLogs) (270) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:56,212::__init__::232::IOProcess::(_processLogs) (270) Queuing response >Thread-16::DEBUG::2014-10-15 06:51:56,212::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:51:56,212::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:51:56,212::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:51:56,212::__init__::232::IOProcess::(_processLogs) (271) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:51:56,212::__init__::232::IOProcess::(_processLogs) (271) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:51:56,955::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:57,777::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:57,778::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-332::DEBUG::2014-10-15 06:51:57,779::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:51:57,780::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:51:57,781::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-333::DEBUG::2014-10-15 06:51:57,782::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-333::DEBUG::2014-10-15 06:51:57,782::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:51:57,956::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:58,018::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:58,019::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:51:58,019::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:51:58,957::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:51:59,959::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:00,805::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:00,806::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-334::DEBUG::2014-10-15 06:52:00,808::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:00,809::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:00,809::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-335::DEBUG::2014-10-15 06:52:00,810::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-335::DEBUG::2014-10-15 06:52:00,811::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:00,960::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:01,961::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:02,963::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:03,022::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:03,022::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:03,023::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 06:52:03,294::task::595::Storage.TaskManager.Task::(_updateState) Task=`3e7d457c-c7e4-4f57-81e6-be828a433194`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 06:52:03,294::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:52:03,295::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:03,295::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:03,295::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:03,295::__init__::232::IOProcess::(_processLogs) (272) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:52:03,296::__init__::232::IOProcess::(_processLogs) (272) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:03,297::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:03,297::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:03,297::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:03,297::__init__::232::IOProcess::(_processLogs) (273) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:52:03,297::__init__::232::IOProcess::(_processLogs) (273) Queuing response >Thread-17::INFO::2014-10-15 06:52:03,297::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:52:03,298::task::1191::Storage.TaskManager.Task::(prepare) Task=`3e7d457c-c7e4-4f57-81e6-be828a433194`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:52:03,298::task::595::Storage.TaskManager.Task::(_updateState) Task=`3e7d457c-c7e4-4f57-81e6-be828a433194`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 06:52:03,298::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 06:52:03,298::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 06:52:03,298::task::993::Storage.TaskManager.Task::(_decref) Task=`3e7d457c-c7e4-4f57-81e6-be828a433194`::ref 0 aborting False >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:03,857::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:03,858::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-336::DEBUG::2014-10-15 06:52:03,859::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:03,860::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:03,860::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-337::DEBUG::2014-10-15 06:52:03,861::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-337::DEBUG::2014-10-15 06:52:03,862::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:03,964::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:04,966::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:05,967::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:52:06,213::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:06,213::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:06,214::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:06,214::__init__::232::IOProcess::(_processLogs) (274) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:52:06,214::__init__::232::IOProcess::(_processLogs) (274) Queuing response >Thread-25::DEBUG::2014-10-15 06:52:06,214::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:52:06,221::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000475861 s, 965 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:52:06,222::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:06,222::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:06,222::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:06,222::__init__::232::IOProcess::(_processLogs) (275) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:52:06,222::__init__::232::IOProcess::(_processLogs) (275) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:06,223::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:06,223::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:06,223::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:06,223::__init__::232::IOProcess::(_processLogs) (276) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:06,223::__init__::232::IOProcess::(_processLogs) (276) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:06,224::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:06,224::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:06,224::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:06,224::__init__::232::IOProcess::(_processLogs) (277) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:06,224::__init__::232::IOProcess::(_processLogs) (277) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:06,224::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:06,225::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:06,225::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:06,225::__init__::232::IOProcess::(_processLogs) (278) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:06,225::__init__::232::IOProcess::(_processLogs) (278) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:06,878::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:06,878::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-338::DEBUG::2014-10-15 06:52:06,880::task::595::Storage.TaskManager.Task::(_updateState) Task=`47f5eb4d-be14-47f8-b3cc-d953a2a9ef7e`::moving from state init -> state preparing >Thread-338::INFO::2014-10-15 06:52:06,880::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-338::INFO::2014-10-15 06:52:06,880::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000475861', 'lastCheck': '0.7', 'valid': True}} >Thread-338::DEBUG::2014-10-15 06:52:06,880::task::1191::Storage.TaskManager.Task::(prepare) Task=`47f5eb4d-be14-47f8-b3cc-d953a2a9ef7e`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000475861', 'lastCheck': '0.7', 'valid': True}} >Thread-338::DEBUG::2014-10-15 06:52:06,880::task::595::Storage.TaskManager.Task::(_updateState) Task=`47f5eb4d-be14-47f8-b3cc-d953a2a9ef7e`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-10-15 06:52:06,880::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-338::DEBUG::2014-10-15 06:52:06,880::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-10-15 06:52:06,881::task::993::Storage.TaskManager.Task::(_decref) Task=`47f5eb4d-be14-47f8-b3cc-d953a2a9ef7e`::ref 0 aborting False >Thread-338::ERROR::2014-10-15 06:52:06,881::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-338::DEBUG::2014-10-15 06:52:06,882::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:06,899::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:06,899::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-339::DEBUG::2014-10-15 06:52:06,900::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-339::DEBUG::2014-10-15 06:52:06,901::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:06,968::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:07,969::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:07,970::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:52:07,970::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:08,029::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:08,029::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:08,029::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:52:08,971::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:09,972::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:10,007::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:10,008::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-340::DEBUG::2014-10-15 06:52:10,009::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:10,010::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:10,011::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-341::DEBUG::2014-10-15 06:52:10,011::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-341::DEBUG::2014-10-15 06:52:10,012::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:10,973::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:11,974::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:12,976::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:13,035::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:13,036::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:13,036::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:13,062::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:13,062::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-342::DEBUG::2014-10-15 06:52:13,063::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:13,064::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:13,065::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-343::DEBUG::2014-10-15 06:52:13,066::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-343::DEBUG::2014-10-15 06:52:13,066::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:13,977::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:14,978::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:15,979::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:16,087::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:16,088::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-344::DEBUG::2014-10-15 06:52:16,089::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:16,090::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:16,091::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-345::DEBUG::2014-10-15 06:52:16,092::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-345::DEBUG::2014-10-15 06:52:16,092::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 06:52:16,226::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:16,226::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:16,226::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:16,226::__init__::232::IOProcess::(_processLogs) (279) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:52:16,226::__init__::232::IOProcess::(_processLogs) (279) Queuing response >Thread-25::DEBUG::2014-10-15 06:52:16,227::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:52:16,232::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000358608 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:52:16,233::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:16,233::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:16,233::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:16,234::__init__::232::IOProcess::(_processLogs) (280) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:52:16,234::__init__::232::IOProcess::(_processLogs) (280) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:16,234::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:16,234::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:16,235::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:16,235::__init__::232::IOProcess::(_processLogs) (281) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:16,235::__init__::232::IOProcess::(_processLogs) (281) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:16,235::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:16,235::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:16,235::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:16,235::__init__::232::IOProcess::(_processLogs) (282) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:16,235::__init__::232::IOProcess::(_processLogs) (282) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:16,236::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:16,236::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:16,236::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:16,236::__init__::232::IOProcess::(_processLogs) (283) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:16,236::__init__::232::IOProcess::(_processLogs) (283) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:52:16,980::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:17,981::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:18,042::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:18,042::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:18,043::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:52:18,983::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:19,143::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:19,143::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-346::DEBUG::2014-10-15 06:52:19,144::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:19,145::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:19,146::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-347::DEBUG::2014-10-15 06:52:19,147::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-347::DEBUG::2014-10-15 06:52:19,147::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:19,984::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:20,986::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:21,987::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:22,167::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:22,167::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-348::DEBUG::2014-10-15 06:52:22,169::task::595::Storage.TaskManager.Task::(_updateState) Task=`1dbed541-30d3-4fae-b407-22f4f9551c3a`::moving from state init -> state preparing >Thread-348::INFO::2014-10-15 06:52:22,169::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-348::INFO::2014-10-15 06:52:22,170::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000358608', 'lastCheck': '5.9', 'valid': True}} >Thread-348::DEBUG::2014-10-15 06:52:22,170::task::1191::Storage.TaskManager.Task::(prepare) Task=`1dbed541-30d3-4fae-b407-22f4f9551c3a`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000358608', 'lastCheck': '5.9', 'valid': True}} >Thread-348::DEBUG::2014-10-15 06:52:22,170::task::595::Storage.TaskManager.Task::(_updateState) Task=`1dbed541-30d3-4fae-b407-22f4f9551c3a`::moving from state preparing -> state finished >Thread-348::DEBUG::2014-10-15 06:52:22,170::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-348::DEBUG::2014-10-15 06:52:22,170::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-348::DEBUG::2014-10-15 06:52:22,170::task::993::Storage.TaskManager.Task::(_decref) Task=`1dbed541-30d3-4fae-b407-22f4f9551c3a`::ref 0 aborting False >Thread-348::ERROR::2014-10-15 06:52:22,171::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-348::DEBUG::2014-10-15 06:52:22,171::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:22,189::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:22,189::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-349::DEBUG::2014-10-15 06:52:22,190::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-349::DEBUG::2014-10-15 06:52:22,191::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:22,988::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:23,048::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:23,049::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:23,049::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:52:23,990::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:24,991::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:25,288::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:25,289::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-350::DEBUG::2014-10-15 06:52:25,290::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:25,291::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:25,291::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-351::DEBUG::2014-10-15 06:52:25,292::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-351::DEBUG::2014-10-15 06:52:25,293::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:25,992::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:52:26,237::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:26,237::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:26,237::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:26,237::__init__::232::IOProcess::(_processLogs) (284) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:52:26,237::__init__::232::IOProcess::(_processLogs) (284) Queuing response >Thread-25::DEBUG::2014-10-15 06:52:26,238::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:52:26,244::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000393647 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:52:26,244::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:26,245::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:26,245::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:26,245::__init__::232::IOProcess::(_processLogs) (285) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:52:26,245::__init__::232::IOProcess::(_processLogs) (285) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:26,245::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:26,245::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:26,246::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:26,246::__init__::232::IOProcess::(_processLogs) (286) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:26,246::__init__::232::IOProcess::(_processLogs) (286) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:26,246::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:26,246::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:26,246::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:26,247::__init__::232::IOProcess::(_processLogs) (287) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:26,247::__init__::232::IOProcess::(_processLogs) (287) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:26,247::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:26,247::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:26,247::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:26,247::__init__::232::IOProcess::(_processLogs) (288) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:26,247::__init__::232::IOProcess::(_processLogs) (288) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:52:26,993::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:27,994::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:28,055::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:28,056::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:28,056::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:28,323::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:28,324::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-352::DEBUG::2014-10-15 06:52:28,325::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:28,326::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:28,326::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-353::DEBUG::2014-10-15 06:52:28,327::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-353::DEBUG::2014-10-15 06:52:28,328::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:28,995::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:29,997::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:30,998::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:31,347::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:31,348::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-354::DEBUG::2014-10-15 06:52:31,349::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:31,350::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:31,351::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-355::DEBUG::2014-10-15 06:52:31,352::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-355::DEBUG::2014-10-15 06:52:31,352::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:31,999::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:33,001::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:33,062::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:33,062::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:33,063::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:52:34,002::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:34,375::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:34,376::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-356::DEBUG::2014-10-15 06:52:34,377::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:34,378::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:34,378::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-357::DEBUG::2014-10-15 06:52:34,379::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-357::DEBUG::2014-10-15 06:52:34,380::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:35,003::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:36,005::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:52:36,248::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:36,249::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:36,249::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:36,249::__init__::232::IOProcess::(_processLogs) (289) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:52:36,249::__init__::232::IOProcess::(_processLogs) (289) Queuing response >Thread-25::DEBUG::2014-10-15 06:52:36,249::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:52:36,255::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000400644 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:52:36,256::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:36,256::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:36,256::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:36,256::__init__::232::IOProcess::(_processLogs) (290) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:52:36,256::__init__::232::IOProcess::(_processLogs) (290) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:36,257::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:36,257::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:36,257::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:36,257::__init__::232::IOProcess::(_processLogs) (291) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:36,257::__init__::232::IOProcess::(_processLogs) (291) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:36,257::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:36,257::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:36,258::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:36,258::__init__::232::IOProcess::(_processLogs) (292) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:36,258::__init__::232::IOProcess::(_processLogs) (292) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:36,258::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:36,258::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:36,258::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:36,258::__init__::232::IOProcess::(_processLogs) (293) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:36,258::__init__::232::IOProcess::(_processLogs) (293) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:52:37,006::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:37,402::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:37,403::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-358::DEBUG::2014-10-15 06:52:37,404::task::595::Storage.TaskManager.Task::(_updateState) Task=`5559ab84-22d1-462c-987e-7897eac36f37`::moving from state init -> state preparing >Thread-358::INFO::2014-10-15 06:52:37,404::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-358::INFO::2014-10-15 06:52:37,404::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000400644', 'lastCheck': '1.1', 'valid': True}} >Thread-358::DEBUG::2014-10-15 06:52:37,404::task::1191::Storage.TaskManager.Task::(prepare) Task=`5559ab84-22d1-462c-987e-7897eac36f37`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000400644', 'lastCheck': '1.1', 'valid': True}} >Thread-358::DEBUG::2014-10-15 06:52:37,404::task::595::Storage.TaskManager.Task::(_updateState) Task=`5559ab84-22d1-462c-987e-7897eac36f37`::moving from state preparing -> state finished >Thread-358::DEBUG::2014-10-15 06:52:37,405::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-358::DEBUG::2014-10-15 06:52:37,405::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-358::DEBUG::2014-10-15 06:52:37,405::task::993::Storage.TaskManager.Task::(_decref) Task=`5559ab84-22d1-462c-987e-7897eac36f37`::ref 0 aborting False >Thread-358::ERROR::2014-10-15 06:52:37,406::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-358::DEBUG::2014-10-15 06:52:37,406::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:37,434::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:37,435::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-359::DEBUG::2014-10-15 06:52:37,436::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-359::DEBUG::2014-10-15 06:52:37,436::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:38,007::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:38,008::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:52:38,008::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:38,069::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:38,070::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:38,070::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:52:39,009::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:40,010::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:40,634::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:40,635::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-360::DEBUG::2014-10-15 06:52:40,636::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:40,637::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:40,637::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-361::DEBUG::2014-10-15 06:52:40,638::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-361::DEBUG::2014-10-15 06:52:40,639::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:41,011::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:42,013::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:43,014::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:43,076::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:43,076::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:43,077::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:43,673::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:43,674::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-362::DEBUG::2014-10-15 06:52:43,674::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:43,675::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:43,676::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-363::DEBUG::2014-10-15 06:52:43,677::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-363::DEBUG::2014-10-15 06:52:43,677::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:44,015::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:45,016::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:46,018::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:52:46,259::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:46,260::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:46,260::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:46,260::__init__::232::IOProcess::(_processLogs) (294) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:52:46,260::__init__::232::IOProcess::(_processLogs) (294) Queuing response >Thread-25::DEBUG::2014-10-15 06:52:46,260::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:52:46,266::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000464457 s, 988 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:52:46,267::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:46,267::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:46,267::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:46,268::__init__::232::IOProcess::(_processLogs) (295) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:52:46,268::__init__::232::IOProcess::(_processLogs) (295) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:46,268::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:46,268::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:46,268::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:46,268::__init__::232::IOProcess::(_processLogs) (296) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:46,269::__init__::232::IOProcess::(_processLogs) (296) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:46,269::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:46,269::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:46,269::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:46,269::__init__::232::IOProcess::(_processLogs) (297) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:46,269::__init__::232::IOProcess::(_processLogs) (297) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:46,270::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:46,270::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:46,270::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:46,270::__init__::232::IOProcess::(_processLogs) (298) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:46,270::__init__::232::IOProcess::(_processLogs) (298) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:46,700::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:46,701::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-364::DEBUG::2014-10-15 06:52:46,702::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:46,703::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:46,703::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-365::DEBUG::2014-10-15 06:52:46,704::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-365::DEBUG::2014-10-15 06:52:46,705::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:47,019::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:48,020::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:48,078::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:48,078::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:48,079::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:52:49,021::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:49,752::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:49,753::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-366::DEBUG::2014-10-15 06:52:49,754::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:49,755::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:49,755::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-367::DEBUG::2014-10-15 06:52:49,756::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-367::DEBUG::2014-10-15 06:52:49,757::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:50,023::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:51,024::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:52,025::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:52,773::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:52,773::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-368::DEBUG::2014-10-15 06:52:52,775::task::595::Storage.TaskManager.Task::(_updateState) Task=`fd02b9e6-2c53-48f3-83fd-ebd6f227505d`::moving from state init -> state preparing >Thread-368::INFO::2014-10-15 06:52:52,775::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-368::INFO::2014-10-15 06:52:52,775::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000464457', 'lastCheck': '6.5', 'valid': True}} >Thread-368::DEBUG::2014-10-15 06:52:52,775::task::1191::Storage.TaskManager.Task::(prepare) Task=`fd02b9e6-2c53-48f3-83fd-ebd6f227505d`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000464457', 'lastCheck': '6.5', 'valid': True}} >Thread-368::DEBUG::2014-10-15 06:52:52,775::task::595::Storage.TaskManager.Task::(_updateState) Task=`fd02b9e6-2c53-48f3-83fd-ebd6f227505d`::moving from state preparing -> state finished >Thread-368::DEBUG::2014-10-15 06:52:52,775::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-368::DEBUG::2014-10-15 06:52:52,775::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-368::DEBUG::2014-10-15 06:52:52,776::task::993::Storage.TaskManager.Task::(_decref) Task=`fd02b9e6-2c53-48f3-83fd-ebd6f227505d`::ref 0 aborting False >Thread-368::ERROR::2014-10-15 06:52:52,776::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-368::DEBUG::2014-10-15 06:52:52,777::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:52,792::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:52,793::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-369::DEBUG::2014-10-15 06:52:52,794::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-369::DEBUG::2014-10-15 06:52:52,794::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:53,027::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:53,084::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:53,085::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:53,085::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:52:54,028::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:55,029::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:55,967::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:55,968::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-370::DEBUG::2014-10-15 06:52:55,969::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:55,970::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:55,970::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-371::DEBUG::2014-10-15 06:52:55,971::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-371::DEBUG::2014-10-15 06:52:55,972::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:56,030::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:52:56,271::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:56,271::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:56,271::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:56,271::__init__::232::IOProcess::(_processLogs) (299) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:52:56,272::__init__::232::IOProcess::(_processLogs) (299) Queuing response >Thread-25::DEBUG::2014-10-15 06:52:56,272::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:52:56,278::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000377487 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:52:56,279::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:56,280::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:56,280::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:56,280::__init__::232::IOProcess::(_processLogs) (300) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:52:56,280::__init__::232::IOProcess::(_processLogs) (300) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:56,281::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:56,281::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:56,281::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:56,281::__init__::232::IOProcess::(_processLogs) (301) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:56,281::__init__::232::IOProcess::(_processLogs) (301) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:56,281::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:56,282::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:56,282::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:56,282::__init__::232::IOProcess::(_processLogs) (302) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:56,282::__init__::232::IOProcess::(_processLogs) (302) Queuing response >Thread-16::DEBUG::2014-10-15 06:52:56,282::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:52:56,282::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:52:56,282::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:52:56,282::__init__::232::IOProcess::(_processLogs) (303) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:52:56,283::__init__::232::IOProcess::(_processLogs) (303) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:52:57,031::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:52:58,033::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:58,086::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:58,087::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:52:58,087::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:59,002::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:59,003::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-372::DEBUG::2014-10-15 06:52:59,004::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:52:59,005::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:52:59,006::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-373::DEBUG::2014-10-15 06:52:59,007::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-373::DEBUG::2014-10-15 06:52:59,007::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:52:59,034::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:00,035::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:01,036::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:02,033::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:02,035::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-374::DEBUG::2014-10-15 06:53:02,038::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:02,038::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:02,041::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:02,043::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-375::DEBUG::2014-10-15 06:53:02,044::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-375::DEBUG::2014-10-15 06:53:02,046::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:03,040::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:03,093::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:03,093::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:03,094::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 06:53:03,336::task::595::Storage.TaskManager.Task::(_updateState) Task=`25cf44d1-01da-4efc-a481-3702585c1373`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 06:53:03,336::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:53:03,337::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:03,337::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:03,337::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:03,337::__init__::232::IOProcess::(_processLogs) (304) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:53:03,338::__init__::232::IOProcess::(_processLogs) (304) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:03,339::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:03,339::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:03,339::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:03,339::__init__::232::IOProcess::(_processLogs) (305) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:53:03,339::__init__::232::IOProcess::(_processLogs) (305) Queuing response >Thread-17::INFO::2014-10-15 06:53:03,339::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:53:03,340::task::1191::Storage.TaskManager.Task::(prepare) Task=`25cf44d1-01da-4efc-a481-3702585c1373`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:53:03,340::task::595::Storage.TaskManager.Task::(_updateState) Task=`25cf44d1-01da-4efc-a481-3702585c1373`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 06:53:03,340::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 06:53:03,340::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 06:53:03,340::task::993::Storage.TaskManager.Task::(_decref) Task=`25cf44d1-01da-4efc-a481-3702585c1373`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:53:04,041::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:05,042::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:05,064::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:05,064::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-376::DEBUG::2014-10-15 06:53:05,065::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:05,066::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:05,067::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-377::DEBUG::2014-10-15 06:53:05,068::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-377::DEBUG::2014-10-15 06:53:05,068::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:06,044::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:53:06,284::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:06,284::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:06,284::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:06,284::__init__::232::IOProcess::(_processLogs) (306) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:53:06,284::__init__::232::IOProcess::(_processLogs) (306) Queuing response >Thread-25::DEBUG::2014-10-15 06:53:06,285::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:53:06,291::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000475053 s, 966 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:53:06,292::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:06,292::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:06,292::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:06,292::__init__::232::IOProcess::(_processLogs) (307) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:53:06,292::__init__::232::IOProcess::(_processLogs) (307) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:06,293::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:06,293::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:06,293::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:06,293::__init__::232::IOProcess::(_processLogs) (308) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:06,293::__init__::232::IOProcess::(_processLogs) (308) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:06,294::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:06,294::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:06,294::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:06,294::__init__::232::IOProcess::(_processLogs) (309) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:06,294::__init__::232::IOProcess::(_processLogs) (309) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:06,294::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:06,294::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:06,295::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:06,295::__init__::232::IOProcess::(_processLogs) (310) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:06,295::__init__::232::IOProcess::(_processLogs) (310) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:53:07,045::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:08,046::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:08,047::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:53:08,047::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:08,092::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:08,092::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-378::DEBUG::2014-10-15 06:53:08,094::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca21cc40-3e92-4e9f-ae79-d45ef2b3f342`::moving from state init -> state preparing >Thread-378::INFO::2014-10-15 06:53:08,094::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-378::INFO::2014-10-15 06:53:08,094::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000475053', 'lastCheck': '1.8', 'valid': True}} >Thread-378::DEBUG::2014-10-15 06:53:08,094::task::1191::Storage.TaskManager.Task::(prepare) Task=`ca21cc40-3e92-4e9f-ae79-d45ef2b3f342`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000475053', 'lastCheck': '1.8', 'valid': True}} >Thread-378::DEBUG::2014-10-15 06:53:08,094::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca21cc40-3e92-4e9f-ae79-d45ef2b3f342`::moving from state preparing -> state finished >Thread-378::DEBUG::2014-10-15 06:53:08,094::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-378::DEBUG::2014-10-15 06:53:08,095::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-378::DEBUG::2014-10-15 06:53:08,095::task::993::Storage.TaskManager.Task::(_decref) Task=`ca21cc40-3e92-4e9f-ae79-d45ef2b3f342`::ref 0 aborting False >Thread-378::ERROR::2014-10-15 06:53:08,095::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-378::DEBUG::2014-10-15 06:53:08,096::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:08,099::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:08,100::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:08,100::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:08,108::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:08,109::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-379::DEBUG::2014-10-15 06:53:08,110::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-379::DEBUG::2014-10-15 06:53:08,110::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:09,048::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:10,048::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:11,049::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:11,217::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:11,218::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-380::DEBUG::2014-10-15 06:53:11,219::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:11,220::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:11,221::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-381::DEBUG::2014-10-15 06:53:11,221::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-381::DEBUG::2014-10-15 06:53:11,222::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:12,050::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:13,051::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:13,106::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:13,107::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:13,107::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:53:14,052::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:14,247::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:14,247::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-382::DEBUG::2014-10-15 06:53:14,248::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:14,249::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:14,250::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-383::DEBUG::2014-10-15 06:53:14,251::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-383::DEBUG::2014-10-15 06:53:14,251::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:15,053::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:16,055::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:53:16,296::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:16,296::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:16,296::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:16,296::__init__::232::IOProcess::(_processLogs) (311) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:53:16,296::__init__::232::IOProcess::(_processLogs) (311) Queuing response >Thread-25::DEBUG::2014-10-15 06:53:16,297::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:53:16,303::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000360821 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:53:16,303::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:16,304::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:16,304::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:16,304::__init__::232::IOProcess::(_processLogs) (312) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:53:16,304::__init__::232::IOProcess::(_processLogs) (312) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:16,304::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:16,305::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:16,305::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:16,305::__init__::232::IOProcess::(_processLogs) (313) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:16,305::__init__::232::IOProcess::(_processLogs) (313) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:16,305::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:16,305::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:16,305::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:16,306::__init__::232::IOProcess::(_processLogs) (314) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:16,306::__init__::232::IOProcess::(_processLogs) (314) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:16,306::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:16,306::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:16,306::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:16,306::__init__::232::IOProcess::(_processLogs) (315) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:16,306::__init__::232::IOProcess::(_processLogs) (315) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:53:17,056::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:17,268::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:17,269::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-384::DEBUG::2014-10-15 06:53:17,270::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:17,270::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:17,271::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-385::DEBUG::2014-10-15 06:53:17,272::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-385::DEBUG::2014-10-15 06:53:17,272::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:18,057::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:18,108::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:18,108::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:18,109::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:53:19,059::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:20,060::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:20,312::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:20,312::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-386::DEBUG::2014-10-15 06:53:20,314::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:20,315::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:20,315::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-387::DEBUG::2014-10-15 06:53:20,316::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-387::DEBUG::2014-10-15 06:53:20,316::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:21,061::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:22,062::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:23,064::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:23,115::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:23,115::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:23,116::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:23,351::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:23,351::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-388::DEBUG::2014-10-15 06:53:23,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`7549c6e9-95fb-4020-8dda-f1bb58d9ce0a`::moving from state init -> state preparing >Thread-388::INFO::2014-10-15 06:53:23,352::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-388::INFO::2014-10-15 06:53:23,353::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000360821', 'lastCheck': '7.0', 'valid': True}} >Thread-388::DEBUG::2014-10-15 06:53:23,353::task::1191::Storage.TaskManager.Task::(prepare) Task=`7549c6e9-95fb-4020-8dda-f1bb58d9ce0a`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000360821', 'lastCheck': '7.0', 'valid': True}} >Thread-388::DEBUG::2014-10-15 06:53:23,353::task::595::Storage.TaskManager.Task::(_updateState) Task=`7549c6e9-95fb-4020-8dda-f1bb58d9ce0a`::moving from state preparing -> state finished >Thread-388::DEBUG::2014-10-15 06:53:23,353::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-388::DEBUG::2014-10-15 06:53:23,353::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-388::DEBUG::2014-10-15 06:53:23,353::task::993::Storage.TaskManager.Task::(_decref) Task=`7549c6e9-95fb-4020-8dda-f1bb58d9ce0a`::ref 0 aborting False >Thread-388::ERROR::2014-10-15 06:53:23,354::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-388::DEBUG::2014-10-15 06:53:23,355::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:23,383::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:23,383::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-389::DEBUG::2014-10-15 06:53:23,384::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-389::DEBUG::2014-10-15 06:53:23,385::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:24,064::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:25,066::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:26,067::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:53:26,307::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:26,307::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:26,308::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:26,308::__init__::232::IOProcess::(_processLogs) (316) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:53:26,308::__init__::232::IOProcess::(_processLogs) (316) Queuing response >Thread-25::DEBUG::2014-10-15 06:53:26,308::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:53:26,314::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00042776 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:53:26,315::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:26,315::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:26,315::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:26,315::__init__::232::IOProcess::(_processLogs) (317) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:53:26,315::__init__::232::IOProcess::(_processLogs) (317) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:26,316::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:26,316::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:26,316::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:26,316::__init__::232::IOProcess::(_processLogs) (318) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:26,316::__init__::232::IOProcess::(_processLogs) (318) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:26,317::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:26,317::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:26,317::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:26,317::__init__::232::IOProcess::(_processLogs) (319) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:26,317::__init__::232::IOProcess::(_processLogs) (319) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:26,317::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:26,317::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:26,318::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:26,318::__init__::232::IOProcess::(_processLogs) (320) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:26,318::__init__::232::IOProcess::(_processLogs) (320) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:26,476::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:26,477::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-390::DEBUG::2014-10-15 06:53:26,478::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:26,479::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:26,480::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-391::DEBUG::2014-10-15 06:53:26,480::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-391::DEBUG::2014-10-15 06:53:26,481::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:27,068::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:28,069::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:28,121::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:28,122::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:28,122::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:53:29,071::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:29,523::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:29,524::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-392::DEBUG::2014-10-15 06:53:29,525::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:29,526::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:29,527::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-393::DEBUG::2014-10-15 06:53:29,528::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-393::DEBUG::2014-10-15 06:53:29,528::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:30,072::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:31,073::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:32,075::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:32,546::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:32,547::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-394::DEBUG::2014-10-15 06:53:32,548::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:32,549::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:32,549::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-395::DEBUG::2014-10-15 06:53:32,550::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-395::DEBUG::2014-10-15 06:53:32,551::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:33,076::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:33,128::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:33,128::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:33,129::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:53:34,077::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:35,079::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:35,575::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:35,575::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-396::DEBUG::2014-10-15 06:53:35,576::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:35,578::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:35,578::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-397::DEBUG::2014-10-15 06:53:35,579::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-397::DEBUG::2014-10-15 06:53:35,579::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:36,080::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:53:36,319::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:36,319::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:36,319::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:36,319::__init__::232::IOProcess::(_processLogs) (321) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:53:36,319::__init__::232::IOProcess::(_processLogs) (321) Queuing response >Thread-25::DEBUG::2014-10-15 06:53:36,319::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:53:36,325::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000367867 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:53:36,326::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:36,326::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:36,327::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:36,327::__init__::232::IOProcess::(_processLogs) (322) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:53:36,327::__init__::232::IOProcess::(_processLogs) (322) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:36,327::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:36,327::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:36,328::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:36,328::__init__::232::IOProcess::(_processLogs) (323) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:36,328::__init__::232::IOProcess::(_processLogs) (323) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:36,328::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:36,328::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:36,328::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:36,328::__init__::232::IOProcess::(_processLogs) (324) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:36,328::__init__::232::IOProcess::(_processLogs) (324) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:36,329::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:36,329::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:36,329::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:36,329::__init__::232::IOProcess::(_processLogs) (325) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:36,329::__init__::232::IOProcess::(_processLogs) (325) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:53:37,081::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:38,083::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:38,083::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:53:38,083::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:38,135::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:38,135::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:38,136::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:38,606::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:38,606::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-398::DEBUG::2014-10-15 06:53:38,607::task::595::Storage.TaskManager.Task::(_updateState) Task=`86e78864-8a1b-4765-9b8b-d8b4f9ac6841`::moving from state init -> state preparing >Thread-398::INFO::2014-10-15 06:53:38,608::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-398::INFO::2014-10-15 06:53:38,608::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000367867', 'lastCheck': '2.3', 'valid': True}} >Thread-398::DEBUG::2014-10-15 06:53:38,608::task::1191::Storage.TaskManager.Task::(prepare) Task=`86e78864-8a1b-4765-9b8b-d8b4f9ac6841`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000367867', 'lastCheck': '2.3', 'valid': True}} >Thread-398::DEBUG::2014-10-15 06:53:38,608::task::595::Storage.TaskManager.Task::(_updateState) Task=`86e78864-8a1b-4765-9b8b-d8b4f9ac6841`::moving from state preparing -> state finished >Thread-398::DEBUG::2014-10-15 06:53:38,608::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-398::DEBUG::2014-10-15 06:53:38,608::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-398::DEBUG::2014-10-15 06:53:38,608::task::993::Storage.TaskManager.Task::(_decref) Task=`86e78864-8a1b-4765-9b8b-d8b4f9ac6841`::ref 0 aborting False >Thread-398::ERROR::2014-10-15 06:53:38,609::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-398::DEBUG::2014-10-15 06:53:38,610::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:38,620::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:38,620::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-399::DEBUG::2014-10-15 06:53:38,621::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-399::DEBUG::2014-10-15 06:53:38,621::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:39,083::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:40,085::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:41,086::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:41,729::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:41,730::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-400::DEBUG::2014-10-15 06:53:41,731::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:41,732::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:41,733::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-401::DEBUG::2014-10-15 06:53:41,734::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-401::DEBUG::2014-10-15 06:53:41,734::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:42,087::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:43,089::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:43,141::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:43,142::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:43,142::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:53:44,090::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:44,764::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:44,765::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-402::DEBUG::2014-10-15 06:53:44,766::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:44,767::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:44,768::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-403::DEBUG::2014-10-15 06:53:44,768::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-403::DEBUG::2014-10-15 06:53:44,769::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:45,091::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:46,092::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:53:46,330::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:46,330::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:46,330::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:46,331::__init__::232::IOProcess::(_processLogs) (326) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:53:46,331::__init__::232::IOProcess::(_processLogs) (326) Queuing response >Thread-25::DEBUG::2014-10-15 06:53:46,331::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:53:46,337::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000358059 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:53:46,338::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:46,338::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:46,338::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:46,338::__init__::232::IOProcess::(_processLogs) (327) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:53:46,338::__init__::232::IOProcess::(_processLogs) (327) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:46,339::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:46,339::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:46,339::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:46,339::__init__::232::IOProcess::(_processLogs) (328) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:46,339::__init__::232::IOProcess::(_processLogs) (328) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:46,340::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:46,340::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:46,340::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:46,340::__init__::232::IOProcess::(_processLogs) (329) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:46,340::__init__::232::IOProcess::(_processLogs) (329) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:46,340::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:46,340::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:46,341::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:46,341::__init__::232::IOProcess::(_processLogs) (330) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:46,341::__init__::232::IOProcess::(_processLogs) (330) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:53:47,094::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:47,811::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:47,811::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-404::DEBUG::2014-10-15 06:53:47,813::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:47,814::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:47,814::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-405::DEBUG::2014-10-15 06:53:47,815::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-405::DEBUG::2014-10-15 06:53:47,816::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:48,095::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:48,148::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:48,148::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:48,149::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:53:49,096::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:50,097::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:50,838::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:50,839::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-406::DEBUG::2014-10-15 06:53:50,840::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:50,841::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:50,842::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-407::DEBUG::2014-10-15 06:53:50,842::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-407::DEBUG::2014-10-15 06:53:50,843::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:51,099::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:52,100::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:53,101::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:53,155::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:53,155::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:53,156::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:53,898::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:53,899::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-408::DEBUG::2014-10-15 06:53:53,900::task::595::Storage.TaskManager.Task::(_updateState) Task=`a66aa74d-bbd4-4b96-8589-04871d1a99f6`::moving from state init -> state preparing >Thread-408::INFO::2014-10-15 06:53:53,900::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-408::INFO::2014-10-15 06:53:53,900::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000358059', 'lastCheck': '7.6', 'valid': True}} >Thread-408::DEBUG::2014-10-15 06:53:53,900::task::1191::Storage.TaskManager.Task::(prepare) Task=`a66aa74d-bbd4-4b96-8589-04871d1a99f6`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000358059', 'lastCheck': '7.6', 'valid': True}} >Thread-408::DEBUG::2014-10-15 06:53:53,900::task::595::Storage.TaskManager.Task::(_updateState) Task=`a66aa74d-bbd4-4b96-8589-04871d1a99f6`::moving from state preparing -> state finished >Thread-408::DEBUG::2014-10-15 06:53:53,901::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-408::DEBUG::2014-10-15 06:53:53,901::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-408::DEBUG::2014-10-15 06:53:53,901::task::993::Storage.TaskManager.Task::(_decref) Task=`a66aa74d-bbd4-4b96-8589-04871d1a99f6`::ref 0 aborting False >Thread-408::ERROR::2014-10-15 06:53:53,902::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-408::DEBUG::2014-10-15 06:53:53,902::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:53,910::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:53,910::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-409::DEBUG::2014-10-15 06:53:53,911::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-409::DEBUG::2014-10-15 06:53:53,912::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:54,102::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:55,104::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:56,105::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:53:56,342::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:56,342::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:56,342::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:56,342::__init__::232::IOProcess::(_processLogs) (331) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:53:56,342::__init__::232::IOProcess::(_processLogs) (331) Queuing response >Thread-25::DEBUG::2014-10-15 06:53:56,343::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:53:56,349::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000391934 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:53:56,349::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:56,349::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:56,349::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:56,349::__init__::232::IOProcess::(_processLogs) (332) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:53:56,350::__init__::232::IOProcess::(_processLogs) (332) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:56,350::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:56,350::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:56,350::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:56,350::__init__::232::IOProcess::(_processLogs) (333) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:56,351::__init__::232::IOProcess::(_processLogs) (333) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:56,351::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:56,351::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:56,351::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:56,351::__init__::232::IOProcess::(_processLogs) (334) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:56,351::__init__::232::IOProcess::(_processLogs) (334) Queuing response >Thread-16::DEBUG::2014-10-15 06:53:56,352::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:53:56,352::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:53:56,352::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:53:56,352::__init__::232::IOProcess::(_processLogs) (335) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:53:56,352::__init__::232::IOProcess::(_processLogs) (335) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:57,029::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:57,029::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-410::DEBUG::2014-10-15 06:53:57,030::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:53:57,032::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:53:57,032::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-411::DEBUG::2014-10-15 06:53:57,033::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-411::DEBUG::2014-10-15 06:53:57,033::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:53:57,106::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:53:58,107::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:58,161::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:58,162::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:53:58,162::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:53:59,109::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:00,066::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:00,067::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-412::DEBUG::2014-10-15 06:54:00,068::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:00,069::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:00,069::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-413::DEBUG::2014-10-15 06:54:00,070::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-413::DEBUG::2014-10-15 06:54:00,071::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:00,109::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:01,111::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:02,112::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:03,093::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:03,094::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-414::DEBUG::2014-10-15 06:54:03,095::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:03,096::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:03,096::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-415::DEBUG::2014-10-15 06:54:03,097::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-415::DEBUG::2014-10-15 06:54:03,098::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:03,113::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:03,164::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:03,164::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:03,165::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 06:54:03,378::task::595::Storage.TaskManager.Task::(_updateState) Task=`e202806f-e77c-4f53-b416-31ba614035e5`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 06:54:03,378::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:54:03,379::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:03,379::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:03,379::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:03,379::__init__::232::IOProcess::(_processLogs) (336) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:54:03,380::__init__::232::IOProcess::(_processLogs) (336) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:03,381::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:03,381::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:03,381::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:03,381::__init__::232::IOProcess::(_processLogs) (337) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:54:03,381::__init__::232::IOProcess::(_processLogs) (337) Queuing response >Thread-17::INFO::2014-10-15 06:54:03,382::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:54:03,382::task::1191::Storage.TaskManager.Task::(prepare) Task=`e202806f-e77c-4f53-b416-31ba614035e5`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:54:03,382::task::595::Storage.TaskManager.Task::(_updateState) Task=`e202806f-e77c-4f53-b416-31ba614035e5`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 06:54:03,382::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 06:54:03,382::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 06:54:03,382::task::993::Storage.TaskManager.Task::(_decref) Task=`e202806f-e77c-4f53-b416-31ba614035e5`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:54:04,114::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:05,116::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:06,117::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:06,120::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:06,121::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-416::DEBUG::2014-10-15 06:54:06,122::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:06,123::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:06,124::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-417::DEBUG::2014-10-15 06:54:06,125::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-417::DEBUG::2014-10-15 06:54:06,125::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 06:54:06,353::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:06,353::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:06,353::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:06,353::__init__::232::IOProcess::(_processLogs) (338) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:54:06,354::__init__::232::IOProcess::(_processLogs) (338) Queuing response >Thread-25::DEBUG::2014-10-15 06:54:06,354::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:54:06,360::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000419256 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:54:06,361::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:06,361::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:06,361::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:06,361::__init__::232::IOProcess::(_processLogs) (339) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:54:06,362::__init__::232::IOProcess::(_processLogs) (339) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:06,362::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:06,362::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:06,362::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:06,363::__init__::232::IOProcess::(_processLogs) (340) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:06,363::__init__::232::IOProcess::(_processLogs) (340) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:06,363::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:06,363::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:06,363::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:06,363::__init__::232::IOProcess::(_processLogs) (341) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:06,363::__init__::232::IOProcess::(_processLogs) (341) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:06,364::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:06,364::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:06,364::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:06,364::__init__::232::IOProcess::(_processLogs) (342) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:06,364::__init__::232::IOProcess::(_processLogs) (342) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:54:07,118::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:08,120::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:08,120::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:54:08,120::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:08,170::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:08,171::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:08,171::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:54:09,121::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:09,147::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:09,148::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-418::DEBUG::2014-10-15 06:54:09,149::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ef48d00-715d-4a21-9f91-0647feb4aa8a`::moving from state init -> state preparing >Thread-418::INFO::2014-10-15 06:54:09,149::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-418::INFO::2014-10-15 06:54:09,149::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419256', 'lastCheck': '2.8', 'valid': True}} >Thread-418::DEBUG::2014-10-15 06:54:09,150::task::1191::Storage.TaskManager.Task::(prepare) Task=`5ef48d00-715d-4a21-9f91-0647feb4aa8a`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419256', 'lastCheck': '2.8', 'valid': True}} >Thread-418::DEBUG::2014-10-15 06:54:09,150::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ef48d00-715d-4a21-9f91-0647feb4aa8a`::moving from state preparing -> state finished >Thread-418::DEBUG::2014-10-15 06:54:09,150::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-418::DEBUG::2014-10-15 06:54:09,150::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-418::DEBUG::2014-10-15 06:54:09,150::task::993::Storage.TaskManager.Task::(_decref) Task=`5ef48d00-715d-4a21-9f91-0647feb4aa8a`::ref 0 aborting False >Thread-418::ERROR::2014-10-15 06:54:09,151::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-418::DEBUG::2014-10-15 06:54:09,151::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:09,161::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:09,162::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-419::DEBUG::2014-10-15 06:54:09,163::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-419::DEBUG::2014-10-15 06:54:09,163::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:10,122::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:11,124::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:12,125::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:12,276::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:12,277::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-420::DEBUG::2014-10-15 06:54:12,278::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:12,279::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:12,279::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-421::DEBUG::2014-10-15 06:54:12,280::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-421::DEBUG::2014-10-15 06:54:12,281::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:13,126::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:13,177::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:13,177::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:13,178::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:54:14,127::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:15,129::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:15,311::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:15,311::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-422::DEBUG::2014-10-15 06:54:15,312::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:15,313::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:15,314::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-423::DEBUG::2014-10-15 06:54:15,315::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-423::DEBUG::2014-10-15 06:54:15,315::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:16,130::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:54:16,365::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:16,365::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:16,365::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:16,365::__init__::232::IOProcess::(_processLogs) (343) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:54:16,366::__init__::232::IOProcess::(_processLogs) (343) Queuing response >Thread-25::DEBUG::2014-10-15 06:54:16,366::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:54:16,372::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000387024 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:54:16,372::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:16,373::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:16,373::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:16,373::__init__::232::IOProcess::(_processLogs) (344) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:54:16,373::__init__::232::IOProcess::(_processLogs) (344) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:16,374::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:16,374::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:16,374::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:16,374::__init__::232::IOProcess::(_processLogs) (345) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:16,374::__init__::232::IOProcess::(_processLogs) (345) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:16,374::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:16,375::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:16,375::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:16,375::__init__::232::IOProcess::(_processLogs) (346) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:16,375::__init__::232::IOProcess::(_processLogs) (346) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:16,375::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:16,375::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:16,375::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:16,375::__init__::232::IOProcess::(_processLogs) (347) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:16,376::__init__::232::IOProcess::(_processLogs) (347) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:54:17,131::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:18,132::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:18,184::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:18,184::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:18,185::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:18,332::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:18,332::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-424::DEBUG::2014-10-15 06:54:18,333::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:18,334::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:18,335::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-425::DEBUG::2014-10-15 06:54:18,336::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-425::DEBUG::2014-10-15 06:54:18,336::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:19,133::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:20,134::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:21,136::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:21,379::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:21,380::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-426::DEBUG::2014-10-15 06:54:21,381::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:21,382::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:21,382::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-427::DEBUG::2014-10-15 06:54:21,383::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-427::DEBUG::2014-10-15 06:54:21,384::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:22,137::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:23,138::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:23,190::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:23,190::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:23,191::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:54:24,139::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:24,405::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:24,406::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-428::DEBUG::2014-10-15 06:54:24,407::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8de99c5-2778-4b39-90d6-3492b77ed6c9`::moving from state init -> state preparing >Thread-428::INFO::2014-10-15 06:54:24,407::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-428::INFO::2014-10-15 06:54:24,408::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000387024', 'lastCheck': '8.0', 'valid': True}} >Thread-428::DEBUG::2014-10-15 06:54:24,408::task::1191::Storage.TaskManager.Task::(prepare) Task=`c8de99c5-2778-4b39-90d6-3492b77ed6c9`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000387024', 'lastCheck': '8.0', 'valid': True}} >Thread-428::DEBUG::2014-10-15 06:54:24,408::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8de99c5-2778-4b39-90d6-3492b77ed6c9`::moving from state preparing -> state finished >Thread-428::DEBUG::2014-10-15 06:54:24,408::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-428::DEBUG::2014-10-15 06:54:24,408::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-428::DEBUG::2014-10-15 06:54:24,408::task::993::Storage.TaskManager.Task::(_decref) Task=`c8de99c5-2778-4b39-90d6-3492b77ed6c9`::ref 0 aborting False >Thread-428::ERROR::2014-10-15 06:54:24,409::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-428::DEBUG::2014-10-15 06:54:24,409::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:24,421::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:24,421::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-429::DEBUG::2014-10-15 06:54:24,422::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-429::DEBUG::2014-10-15 06:54:24,422::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:25,141::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:26,142::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-25::DEBUG::2014-10-15 06:54:26,376::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 06:54:26,377::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:26,377::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:26,377::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:26,378::__init__::232::IOProcess::(_processLogs) (348) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:54:26,378::__init__::232::IOProcess::(_processLogs) (348) Queuing response >Thread-25::DEBUG::2014-10-15 06:54:26,385::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 06:54:26,385::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:26,385::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:26,385::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:26,385::__init__::232::IOProcess::(_processLogs) (349) Got request for method 'touch' >Thread-16::DEBUG::2014-10-15 06:54:26,412::__init__::232::IOProcess::(_processLogs) (349) Queuing response >Thread-25::DEBUG::2014-10-15 06:54:26,412::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-16::DEBUG::2014-10-15 06:54:26,412::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:26,412::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:26,413::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:26,413::__init__::232::IOProcess::(_processLogs) (350) Got request for method 'readfile' >Thread-16::DEBUG::2014-10-15 06:54:26,413::__init__::232::IOProcess::(_processLogs) (350) Queuing response >Thread-25::DEBUG::2014-10-15 06:54:26,413::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=domain1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-0000000003e1', 'REMOTE_PATH=10.66.128.104:/home/NFS1', 'ROLE=Master', 'SDUUID=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b90f33781d11f6e6b0b3945de3167c0719a458c1'] >Thread-16::DEBUG::2014-10-15 06:54:26,414::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:26,414::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:26,414::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:26,414::__init__::232::IOProcess::(_processLogs) (351) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:26,414::__init__::232::IOProcess::(_processLogs) (351) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:26,415::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:26,415::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:26,415::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:26,415::__init__::232::IOProcess::(_processLogs) (352) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:54:26,415::__init__::232::IOProcess::(_processLogs) (352) Queuing response >Thread-25::DEBUG::2014-10-15 06:54:26,415::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-25::INFO::2014-10-15 06:54:26,415::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_imageNS already registered >Thread-25::INFO::2014-10-15 06:54:26,415::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_volumeNS already registered >Thread-16::DEBUG::2014-10-15 06:54:26,416::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:26,416::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:26,416::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:26,416::__init__::232::IOProcess::(_processLogs) (353) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:54:26,416::__init__::232::IOProcess::(_processLogs) (353) Queuing response >Thread-25::DEBUG::2014-10-15 06:54:26,416::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:54:26,422::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000418029 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:54:26,423::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:26,423::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:26,423::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:26,423::__init__::232::IOProcess::(_processLogs) (354) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:54:26,424::__init__::232::IOProcess::(_processLogs) (354) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:26,424::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:26,424::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:26,424::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:26,424::__init__::232::IOProcess::(_processLogs) (355) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:26,424::__init__::232::IOProcess::(_processLogs) (355) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:26,425::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:26,425::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:26,425::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:26,425::__init__::232::IOProcess::(_processLogs) (356) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:26,425::__init__::232::IOProcess::(_processLogs) (356) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:26,426::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:26,426::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:26,426::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:26,426::__init__::232::IOProcess::(_processLogs) (357) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:26,426::__init__::232::IOProcess::(_processLogs) (357) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:54:27,143::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:27,531::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:27,532::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-431::DEBUG::2014-10-15 06:54:27,533::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:27,534::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:27,535::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-432::DEBUG::2014-10-15 06:54:27,536::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-432::DEBUG::2014-10-15 06:54:27,536::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:28,144::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:28,196::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:28,197::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:28,197::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:54:29,146::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:30,147::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:30,604::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:30,605::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-433::DEBUG::2014-10-15 06:54:30,606::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:30,607::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:30,607::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-434::DEBUG::2014-10-15 06:54:30,608::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-434::DEBUG::2014-10-15 06:54:30,609::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:31,148::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:32,149::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:33,150::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:33,203::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:33,203::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:33,204::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:33,628::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:33,628::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-435::DEBUG::2014-10-15 06:54:33,630::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:33,631::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:33,631::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-436::DEBUG::2014-10-15 06:54:33,632::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-436::DEBUG::2014-10-15 06:54:33,632::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:34,152::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:35,153::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:36,154::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:54:36,427::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:36,427::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:36,427::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:36,427::__init__::232::IOProcess::(_processLogs) (358) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:54:36,428::__init__::232::IOProcess::(_processLogs) (358) Queuing response >Thread-25::DEBUG::2014-10-15 06:54:36,428::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:54:36,434::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000393523 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:54:36,435::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:36,435::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:36,435::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:36,435::__init__::232::IOProcess::(_processLogs) (359) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:54:36,435::__init__::232::IOProcess::(_processLogs) (359) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:36,436::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:36,436::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:36,436::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:36,436::__init__::232::IOProcess::(_processLogs) (360) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:36,436::__init__::232::IOProcess::(_processLogs) (360) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:36,437::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:36,437::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:36,437::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:36,437::__init__::232::IOProcess::(_processLogs) (361) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:36,437::__init__::232::IOProcess::(_processLogs) (361) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:36,437::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:36,437::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:36,437::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:36,438::__init__::232::IOProcess::(_processLogs) (362) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:36,438::__init__::232::IOProcess::(_processLogs) (362) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:36,652::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:36,653::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-437::DEBUG::2014-10-15 06:54:36,654::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:36,655::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:36,656::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-438::DEBUG::2014-10-15 06:54:36,657::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-438::DEBUG::2014-10-15 06:54:36,657::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:37,156::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:38,157::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:38,157::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:54:38,157::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:38,209::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:38,210::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:38,210::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:54:39,158::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:39,676::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:39,677::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-439::DEBUG::2014-10-15 06:54:39,678::task::595::Storage.TaskManager.Task::(_updateState) Task=`1cf10a57-40d0-4f5b-8115-4f56ee919e50`::moving from state init -> state preparing >Thread-439::INFO::2014-10-15 06:54:39,678::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-439::INFO::2014-10-15 06:54:39,678::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000393523', 'lastCheck': '3.2', 'valid': True}} >Thread-439::DEBUG::2014-10-15 06:54:39,678::task::1191::Storage.TaskManager.Task::(prepare) Task=`1cf10a57-40d0-4f5b-8115-4f56ee919e50`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000393523', 'lastCheck': '3.2', 'valid': True}} >Thread-439::DEBUG::2014-10-15 06:54:39,679::task::595::Storage.TaskManager.Task::(_updateState) Task=`1cf10a57-40d0-4f5b-8115-4f56ee919e50`::moving from state preparing -> state finished >Thread-439::DEBUG::2014-10-15 06:54:39,679::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-439::DEBUG::2014-10-15 06:54:39,679::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-439::DEBUG::2014-10-15 06:54:39,679::task::993::Storage.TaskManager.Task::(_decref) Task=`1cf10a57-40d0-4f5b-8115-4f56ee919e50`::ref 0 aborting False >Thread-439::ERROR::2014-10-15 06:54:39,680::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-439::DEBUG::2014-10-15 06:54:39,680::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:39,692::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:39,692::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-440::DEBUG::2014-10-15 06:54:39,693::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-440::DEBUG::2014-10-15 06:54:39,694::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:40,160::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:41,161::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:42,162::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:42,801::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:42,802::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-441::DEBUG::2014-10-15 06:54:42,803::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:42,804::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:42,804::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-442::DEBUG::2014-10-15 06:54:42,805::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-442::DEBUG::2014-10-15 06:54:42,806::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:43,163::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:43,216::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:43,216::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:43,217::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:54:44,165::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:45,166::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:45,838::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:45,839::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-443::DEBUG::2014-10-15 06:54:45,840::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:45,841::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:45,842::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-444::DEBUG::2014-10-15 06:54:45,842::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-444::DEBUG::2014-10-15 06:54:45,843::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:46,167::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:54:46,439::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:46,439::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:46,439::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:46,439::__init__::232::IOProcess::(_processLogs) (363) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:54:46,439::__init__::232::IOProcess::(_processLogs) (363) Queuing response >Thread-25::DEBUG::2014-10-15 06:54:46,439::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:54:46,445::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000357305 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:54:46,446::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:46,446::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:46,446::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:46,446::__init__::232::IOProcess::(_processLogs) (364) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:54:46,447::__init__::232::IOProcess::(_processLogs) (364) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:46,447::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:46,447::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:46,447::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:46,447::__init__::232::IOProcess::(_processLogs) (365) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:46,448::__init__::232::IOProcess::(_processLogs) (365) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:46,448::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:46,448::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:46,448::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:46,448::__init__::232::IOProcess::(_processLogs) (366) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:46,448::__init__::232::IOProcess::(_processLogs) (366) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:46,449::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:46,449::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:46,449::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:46,449::__init__::232::IOProcess::(_processLogs) (367) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:46,449::__init__::232::IOProcess::(_processLogs) (367) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:54:47,168::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:48,170::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:48,221::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:48,221::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:48,222::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:48,867::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:48,867::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-445::DEBUG::2014-10-15 06:54:48,868::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:48,870::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:48,870::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-446::DEBUG::2014-10-15 06:54:48,871::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-446::DEBUG::2014-10-15 06:54:48,871::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:49,171::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:50,172::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:51,173::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:51,913::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:51,913::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-447::DEBUG::2014-10-15 06:54:51,914::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:51,915::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:51,916::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-448::DEBUG::2014-10-15 06:54:51,917::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-448::DEBUG::2014-10-15 06:54:51,917::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:52,175::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:53,176::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:53,227::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:53,228::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:53,228::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:54:54,177::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:54,933::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:54,934::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-449::DEBUG::2014-10-15 06:54:54,935::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf149d27-bfad-41fb-8c52-c1c6ef904745`::moving from state init -> state preparing >Thread-449::INFO::2014-10-15 06:54:54,935::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-449::INFO::2014-10-15 06:54:54,935::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000357305', 'lastCheck': '8.5', 'valid': True}} >Thread-449::DEBUG::2014-10-15 06:54:54,935::task::1191::Storage.TaskManager.Task::(prepare) Task=`cf149d27-bfad-41fb-8c52-c1c6ef904745`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000357305', 'lastCheck': '8.5', 'valid': True}} >Thread-449::DEBUG::2014-10-15 06:54:54,935::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf149d27-bfad-41fb-8c52-c1c6ef904745`::moving from state preparing -> state finished >Thread-449::DEBUG::2014-10-15 06:54:54,936::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-449::DEBUG::2014-10-15 06:54:54,936::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-449::DEBUG::2014-10-15 06:54:54,936::task::993::Storage.TaskManager.Task::(_decref) Task=`cf149d27-bfad-41fb-8c52-c1c6ef904745`::ref 0 aborting False >Thread-449::ERROR::2014-10-15 06:54:54,937::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-449::DEBUG::2014-10-15 06:54:54,937::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:54,949::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:54,950::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-450::DEBUG::2014-10-15 06:54:54,951::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-450::DEBUG::2014-10-15 06:54:54,951::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:55,178::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:54:56,179::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:54:56,450::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:56,450::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:56,450::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:56,450::__init__::232::IOProcess::(_processLogs) (368) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:54:56,451::__init__::232::IOProcess::(_processLogs) (368) Queuing response >Thread-25::DEBUG::2014-10-15 06:54:56,451::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:54:56,457::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000388441 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:54:56,458::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:56,458::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:56,458::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:56,458::__init__::232::IOProcess::(_processLogs) (369) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:54:56,458::__init__::232::IOProcess::(_processLogs) (369) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:56,459::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:56,459::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:56,459::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:56,459::__init__::232::IOProcess::(_processLogs) (370) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:56,459::__init__::232::IOProcess::(_processLogs) (370) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:56,459::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:56,460::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:56,460::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:56,460::__init__::232::IOProcess::(_processLogs) (371) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:56,460::__init__::232::IOProcess::(_processLogs) (371) Queuing response >Thread-16::DEBUG::2014-10-15 06:54:56,460::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:54:56,460::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:54:56,460::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:54:56,460::__init__::232::IOProcess::(_processLogs) (372) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:54:56,461::__init__::232::IOProcess::(_processLogs) (372) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:54:57,181::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:58,075::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:58,075::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-451::DEBUG::2014-10-15 06:54:58,076::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:54:58,077::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:54:58,078::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-452::DEBUG::2014-10-15 06:54:58,079::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-452::DEBUG::2014-10-15 06:54:58,079::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:54:58,182::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:58,234::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:58,234::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:54:58,235::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:54:59,183::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:00,184::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:01,110::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:01,111::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-453::DEBUG::2014-10-15 06:55:01,113::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:01,114::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:01,115::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-454::DEBUG::2014-10-15 06:55:01,116::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-454::DEBUG::2014-10-15 06:55:01,116::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:01,186::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:02,187::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:03,188::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:03,241::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:03,241::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:03,242::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 06:55:03,420::task::595::Storage.TaskManager.Task::(_updateState) Task=`84a3da95-5932-4187-bb47-edf0976f26a9`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 06:55:03,420::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:55:03,420::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:03,420::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:03,420::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:03,421::__init__::232::IOProcess::(_processLogs) (373) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:55:03,422::__init__::232::IOProcess::(_processLogs) (373) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:03,422::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:03,422::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:03,422::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:03,423::__init__::232::IOProcess::(_processLogs) (374) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:55:03,423::__init__::232::IOProcess::(_processLogs) (374) Queuing response >Thread-17::INFO::2014-10-15 06:55:03,423::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:55:03,423::task::1191::Storage.TaskManager.Task::(prepare) Task=`84a3da95-5932-4187-bb47-edf0976f26a9`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:55:03,423::task::595::Storage.TaskManager.Task::(_updateState) Task=`84a3da95-5932-4187-bb47-edf0976f26a9`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 06:55:03,423::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 06:55:03,423::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 06:55:03,423::task::993::Storage.TaskManager.Task::(_decref) Task=`84a3da95-5932-4187-bb47-edf0976f26a9`::ref 0 aborting False >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:04,142::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:04,143::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-455::DEBUG::2014-10-15 06:55:04,144::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:04,145::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:04,145::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-456::DEBUG::2014-10-15 06:55:04,146::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-456::DEBUG::2014-10-15 06:55:04,146::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:04,190::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:05,191::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:06,192::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:55:06,462::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:06,462::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:06,462::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:06,462::__init__::232::IOProcess::(_processLogs) (375) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:55:06,462::__init__::232::IOProcess::(_processLogs) (375) Queuing response >Thread-25::DEBUG::2014-10-15 06:55:06,462::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:55:06,469::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000376445 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:55:06,470::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:06,470::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:06,470::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:06,470::__init__::232::IOProcess::(_processLogs) (376) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:55:06,470::__init__::232::IOProcess::(_processLogs) (376) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:06,471::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:06,471::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:06,471::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:06,471::__init__::232::IOProcess::(_processLogs) (377) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:06,471::__init__::232::IOProcess::(_processLogs) (377) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:06,471::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:06,472::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:06,472::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:06,472::__init__::232::IOProcess::(_processLogs) (378) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:06,472::__init__::232::IOProcess::(_processLogs) (378) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:06,472::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:06,472::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:06,472::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:06,472::__init__::232::IOProcess::(_processLogs) (379) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:06,473::__init__::232::IOProcess::(_processLogs) (379) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:07,171::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:07,171::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-457::DEBUG::2014-10-15 06:55:07,172::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:07,174::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:07,174::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-458::DEBUG::2014-10-15 06:55:07,175::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-458::DEBUG::2014-10-15 06:55:07,175::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:07,192::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:08,194::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:08,194::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:55:08,194::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:08,246::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:08,246::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:08,247::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:55:09,195::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:10,196::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:10,214::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:10,214::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-459::DEBUG::2014-10-15 06:55:10,216::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b97fca4-ec48-4f45-9375-a40033892931`::moving from state init -> state preparing >Thread-459::INFO::2014-10-15 06:55:10,216::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-459::INFO::2014-10-15 06:55:10,216::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000376445', 'lastCheck': '3.7', 'valid': True}} >Thread-459::DEBUG::2014-10-15 06:55:10,216::task::1191::Storage.TaskManager.Task::(prepare) Task=`7b97fca4-ec48-4f45-9375-a40033892931`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000376445', 'lastCheck': '3.7', 'valid': True}} >Thread-459::DEBUG::2014-10-15 06:55:10,216::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b97fca4-ec48-4f45-9375-a40033892931`::moving from state preparing -> state finished >Thread-459::DEBUG::2014-10-15 06:55:10,216::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-459::DEBUG::2014-10-15 06:55:10,216::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-459::DEBUG::2014-10-15 06:55:10,217::task::993::Storage.TaskManager.Task::(_decref) Task=`7b97fca4-ec48-4f45-9375-a40033892931`::ref 0 aborting False >Thread-459::ERROR::2014-10-15 06:55:10,217::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-459::DEBUG::2014-10-15 06:55:10,218::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:10,230::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:10,230::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-460::DEBUG::2014-10-15 06:55:10,231::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-460::DEBUG::2014-10-15 06:55:10,232::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:11,197::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:12,199::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:13,200::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:13,249::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:13,249::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:13,250::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:13,343::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:13,344::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-461::DEBUG::2014-10-15 06:55:13,345::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:13,346::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:13,347::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-462::DEBUG::2014-10-15 06:55:13,348::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-462::DEBUG::2014-10-15 06:55:13,348::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:14,201::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:15,202::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:16,204::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:16,379::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:16,380::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-463::DEBUG::2014-10-15 06:55:16,381::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:16,382::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:16,383::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-464::DEBUG::2014-10-15 06:55:16,384::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-464::DEBUG::2014-10-15 06:55:16,384::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 06:55:16,473::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:16,474::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:16,474::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:16,474::__init__::232::IOProcess::(_processLogs) (380) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:55:16,474::__init__::232::IOProcess::(_processLogs) (380) Queuing response >Thread-25::DEBUG::2014-10-15 06:55:16,474::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:55:16,480::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00037677 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:55:16,481::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:16,481::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:16,481::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:16,481::__init__::232::IOProcess::(_processLogs) (381) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:55:16,481::__init__::232::IOProcess::(_processLogs) (381) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:16,482::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:16,482::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:16,482::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:16,482::__init__::232::IOProcess::(_processLogs) (382) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:16,482::__init__::232::IOProcess::(_processLogs) (382) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:16,483::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:16,483::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:16,483::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:16,483::__init__::232::IOProcess::(_processLogs) (383) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:16,483::__init__::232::IOProcess::(_processLogs) (383) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:16,483::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:16,484::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:16,484::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:16,484::__init__::232::IOProcess::(_processLogs) (384) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:16,484::__init__::232::IOProcess::(_processLogs) (384) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:55:17,205::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:18,206::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:18,255::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:18,256::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:18,256::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:55:19,207::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:19,405::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:19,406::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-465::DEBUG::2014-10-15 06:55:19,407::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:19,408::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:19,409::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-466::DEBUG::2014-10-15 06:55:19,409::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-466::DEBUG::2014-10-15 06:55:19,410::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:20,209::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:21,209::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:22,210::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:22,426::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:22,426::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-467::DEBUG::2014-10-15 06:55:22,427::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:22,428::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:22,429::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-468::DEBUG::2014-10-15 06:55:22,430::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-468::DEBUG::2014-10-15 06:55:22,430::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:23,212::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:23,257::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:23,257::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:23,258::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:55:24,213::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:25,214::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:25,449::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:25,450::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-469::DEBUG::2014-10-15 06:55:25,451::task::595::Storage.TaskManager.Task::(_updateState) Task=`d501220a-fc4f-4cb6-9655-f0653c529e02`::moving from state init -> state preparing >Thread-469::INFO::2014-10-15 06:55:25,451::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-469::INFO::2014-10-15 06:55:25,451::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00037677', 'lastCheck': '9.0', 'valid': True}} >Thread-469::DEBUG::2014-10-15 06:55:25,452::task::1191::Storage.TaskManager.Task::(prepare) Task=`d501220a-fc4f-4cb6-9655-f0653c529e02`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00037677', 'lastCheck': '9.0', 'valid': True}} >Thread-469::DEBUG::2014-10-15 06:55:25,452::task::595::Storage.TaskManager.Task::(_updateState) Task=`d501220a-fc4f-4cb6-9655-f0653c529e02`::moving from state preparing -> state finished >Thread-469::DEBUG::2014-10-15 06:55:25,452::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-469::DEBUG::2014-10-15 06:55:25,452::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-469::DEBUG::2014-10-15 06:55:25,452::task::993::Storage.TaskManager.Task::(_decref) Task=`d501220a-fc4f-4cb6-9655-f0653c529e02`::ref 0 aborting False >Thread-469::ERROR::2014-10-15 06:55:25,453::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-469::DEBUG::2014-10-15 06:55:25,453::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:25,465::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:25,465::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-470::DEBUG::2014-10-15 06:55:25,466::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-470::DEBUG::2014-10-15 06:55:25,467::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:26,215::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:55:26,485::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:26,485::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:26,485::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:26,485::__init__::232::IOProcess::(_processLogs) (385) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:55:26,485::__init__::232::IOProcess::(_processLogs) (385) Queuing response >Thread-25::DEBUG::2014-10-15 06:55:26,486::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:55:26,492::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000378192 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:55:26,492::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:26,492::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:26,493::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:26,493::__init__::232::IOProcess::(_processLogs) (386) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:55:26,493::__init__::232::IOProcess::(_processLogs) (386) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:26,493::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:26,493::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:26,494::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:26,494::__init__::232::IOProcess::(_processLogs) (387) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:26,494::__init__::232::IOProcess::(_processLogs) (387) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:26,494::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:26,494::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:26,494::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:26,494::__init__::232::IOProcess::(_processLogs) (388) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:26,494::__init__::232::IOProcess::(_processLogs) (388) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:26,495::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:26,495::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:26,495::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:26,495::__init__::232::IOProcess::(_processLogs) (389) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:26,495::__init__::232::IOProcess::(_processLogs) (389) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:55:27,217::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:28,218::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:28,260::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:28,261::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:28,261::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:28,568::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:28,569::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-471::DEBUG::2014-10-15 06:55:28,570::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:28,570::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:28,571::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-472::DEBUG::2014-10-15 06:55:28,572::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-472::DEBUG::2014-10-15 06:55:28,572::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:29,219::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:30,220::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:31,221::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:31,620::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:31,621::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-473::DEBUG::2014-10-15 06:55:31,624::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:31,626::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:31,627::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-474::DEBUG::2014-10-15 06:55:31,629::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-474::DEBUG::2014-10-15 06:55:31,631::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:32,223::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:33,224::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:33,267::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:33,267::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:33,268::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:55:34,225::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:34,653::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:34,654::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-475::DEBUG::2014-10-15 06:55:34,655::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:34,656::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:34,656::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-476::DEBUG::2014-10-15 06:55:34,657::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-476::DEBUG::2014-10-15 06:55:34,657::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:35,227::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:36,228::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:55:36,496::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:36,496::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:36,497::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:36,497::__init__::232::IOProcess::(_processLogs) (390) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:55:36,497::__init__::232::IOProcess::(_processLogs) (390) Queuing response >Thread-25::DEBUG::2014-10-15 06:55:36,497::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:55:36,503::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000391583 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:55:36,504::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:36,504::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:36,504::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:36,504::__init__::232::IOProcess::(_processLogs) (391) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:55:36,504::__init__::232::IOProcess::(_processLogs) (391) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:36,505::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:36,505::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:36,505::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:36,505::__init__::232::IOProcess::(_processLogs) (392) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:36,505::__init__::232::IOProcess::(_processLogs) (392) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:36,506::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:36,506::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:36,506::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:36,506::__init__::232::IOProcess::(_processLogs) (393) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:36,506::__init__::232::IOProcess::(_processLogs) (393) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:36,506::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:36,507::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:36,507::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:36,507::__init__::232::IOProcess::(_processLogs) (394) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:36,507::__init__::232::IOProcess::(_processLogs) (394) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:55:37,229::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:37,681::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:37,682::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-477::DEBUG::2014-10-15 06:55:37,683::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:37,684::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:37,685::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-478::DEBUG::2014-10-15 06:55:37,685::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-478::DEBUG::2014-10-15 06:55:37,686::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:38,230::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:38,231::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:55:38,231::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:38,274::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:38,274::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:38,274::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:55:39,232::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:40,233::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:40,719::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:40,719::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-479::DEBUG::2014-10-15 06:55:40,721::task::595::Storage.TaskManager.Task::(_updateState) Task=`a71124cf-e5a3-459a-bb11-02750f1b695b`::moving from state init -> state preparing >Thread-479::INFO::2014-10-15 06:55:40,721::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-479::INFO::2014-10-15 06:55:40,721::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000391583', 'lastCheck': '4.2', 'valid': True}} >Thread-479::DEBUG::2014-10-15 06:55:40,721::task::1191::Storage.TaskManager.Task::(prepare) Task=`a71124cf-e5a3-459a-bb11-02750f1b695b`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000391583', 'lastCheck': '4.2', 'valid': True}} >Thread-479::DEBUG::2014-10-15 06:55:40,721::task::595::Storage.TaskManager.Task::(_updateState) Task=`a71124cf-e5a3-459a-bb11-02750f1b695b`::moving from state preparing -> state finished >Thread-479::DEBUG::2014-10-15 06:55:40,721::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-479::DEBUG::2014-10-15 06:55:40,721::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-479::DEBUG::2014-10-15 06:55:40,722::task::993::Storage.TaskManager.Task::(_decref) Task=`a71124cf-e5a3-459a-bb11-02750f1b695b`::ref 0 aborting False >Thread-479::ERROR::2014-10-15 06:55:40,722::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-479::DEBUG::2014-10-15 06:55:40,723::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:40,735::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:40,736::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-480::DEBUG::2014-10-15 06:55:40,737::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-480::DEBUG::2014-10-15 06:55:40,737::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:41,235::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:42,236::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:43,237::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:43,280::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:43,281::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:43,281::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:43,835::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:43,836::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-481::DEBUG::2014-10-15 06:55:43,837::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:43,838::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:43,839::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-482::DEBUG::2014-10-15 06:55:43,840::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-482::DEBUG::2014-10-15 06:55:43,840::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:44,238::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:45,240::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:46,241::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:55:46,508::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:46,508::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:46,508::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:46,508::__init__::232::IOProcess::(_processLogs) (395) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:55:46,508::__init__::232::IOProcess::(_processLogs) (395) Queuing response >Thread-25::DEBUG::2014-10-15 06:55:46,509::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:55:46,515::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000370168 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:55:46,515::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:46,515::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:46,516::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:46,516::__init__::232::IOProcess::(_processLogs) (396) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:55:46,516::__init__::232::IOProcess::(_processLogs) (396) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:46,516::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:46,517::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:46,517::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:46,517::__init__::232::IOProcess::(_processLogs) (397) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:46,517::__init__::232::IOProcess::(_processLogs) (397) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:46,517::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:46,517::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:46,517::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:46,518::__init__::232::IOProcess::(_processLogs) (398) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:46,518::__init__::232::IOProcess::(_processLogs) (398) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:46,518::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:46,518::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:46,518::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:46,518::__init__::232::IOProcess::(_processLogs) (399) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:46,518::__init__::232::IOProcess::(_processLogs) (399) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:46,901::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:46,902::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-483::DEBUG::2014-10-15 06:55:46,903::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:46,904::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:46,905::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-484::DEBUG::2014-10-15 06:55:46,906::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-484::DEBUG::2014-10-15 06:55:46,906::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:47,242::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:48,243::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:48,284::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:48,284::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:48,285::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:55:49,245::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:49,930::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:49,931::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-485::DEBUG::2014-10-15 06:55:49,932::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:49,933::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:49,933::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-486::DEBUG::2014-10-15 06:55:49,934::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-486::DEBUG::2014-10-15 06:55:49,935::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:50,246::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:51,247::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:52,248::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:52,956::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:52,957::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-487::DEBUG::2014-10-15 06:55:52,958::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:52,959::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:52,959::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-488::DEBUG::2014-10-15 06:55:52,960::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-488::DEBUG::2014-10-15 06:55:52,960::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:53,250::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:53,288::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:53,292::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:53,293::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:55:54,251::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:55,252::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:55,983::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:55,983::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-489::DEBUG::2014-10-15 06:55:55,985::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b315537-303b-4bd2-a3e5-6a2e50c654e0`::moving from state init -> state preparing >Thread-489::INFO::2014-10-15 06:55:55,985::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-489::INFO::2014-10-15 06:55:55,985::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000370168', 'lastCheck': '9.5', 'valid': True}} >Thread-489::DEBUG::2014-10-15 06:55:55,985::task::1191::Storage.TaskManager.Task::(prepare) Task=`1b315537-303b-4bd2-a3e5-6a2e50c654e0`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000370168', 'lastCheck': '9.5', 'valid': True}} >Thread-489::DEBUG::2014-10-15 06:55:55,985::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b315537-303b-4bd2-a3e5-6a2e50c654e0`::moving from state preparing -> state finished >Thread-489::DEBUG::2014-10-15 06:55:55,985::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-489::DEBUG::2014-10-15 06:55:55,985::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-489::DEBUG::2014-10-15 06:55:55,986::task::993::Storage.TaskManager.Task::(_decref) Task=`1b315537-303b-4bd2-a3e5-6a2e50c654e0`::ref 0 aborting False >Thread-489::ERROR::2014-10-15 06:55:55,986::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-489::DEBUG::2014-10-15 06:55:55,987::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:55,997::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:55,998::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-490::DEBUG::2014-10-15 06:55:55,999::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-490::DEBUG::2014-10-15 06:55:55,999::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:56,253::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:55:56,519::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:56,520::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:56,520::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:56,520::__init__::232::IOProcess::(_processLogs) (400) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:55:56,520::__init__::232::IOProcess::(_processLogs) (400) Queuing response >Thread-25::DEBUG::2014-10-15 06:55:56,520::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:55:56,526::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000438418 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:55:56,527::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:56,527::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:56,527::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:56,527::__init__::232::IOProcess::(_processLogs) (401) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:55:56,527::__init__::232::IOProcess::(_processLogs) (401) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:56,528::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:56,528::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:56,528::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:56,528::__init__::232::IOProcess::(_processLogs) (402) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:56,528::__init__::232::IOProcess::(_processLogs) (402) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:56,529::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:56,529::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:56,529::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:56,529::__init__::232::IOProcess::(_processLogs) (403) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:56,529::__init__::232::IOProcess::(_processLogs) (403) Queuing response >Thread-16::DEBUG::2014-10-15 06:55:56,529::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:55:56,530::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:55:56,530::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:55:56,530::__init__::232::IOProcess::(_processLogs) (404) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:55:56,530::__init__::232::IOProcess::(_processLogs) (404) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:55:57,255::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:55:58,256::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:58,296::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:58,297::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:55:58,297::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:59,123::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:59,124::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-491::DEBUG::2014-10-15 06:55:59,125::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:55:59,126::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:55:59,126::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-492::DEBUG::2014-10-15 06:55:59,127::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-492::DEBUG::2014-10-15 06:55:59,128::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:55:59,257::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:00,258::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:01,260::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:02,170::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:02,171::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-493::DEBUG::2014-10-15 06:56:02,174::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:02,176::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:02,177::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-494::DEBUG::2014-10-15 06:56:02,179::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-494::DEBUG::2014-10-15 06:56:02,180::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:02,261::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:03,262::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:03,301::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:03,301::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:03,302::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 06:56:03,462::task::595::Storage.TaskManager.Task::(_updateState) Task=`48df1d66-6ebf-4454-9583-3969b21e4c26`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 06:56:03,462::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:56:03,462::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:03,462::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:03,463::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:03,463::__init__::232::IOProcess::(_processLogs) (405) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:56:03,464::__init__::232::IOProcess::(_processLogs) (405) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:03,464::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:03,464::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:03,465::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:03,465::__init__::232::IOProcess::(_processLogs) (406) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:56:03,465::__init__::232::IOProcess::(_processLogs) (406) Queuing response >Thread-17::INFO::2014-10-15 06:56:03,465::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:56:03,465::task::1191::Storage.TaskManager.Task::(prepare) Task=`48df1d66-6ebf-4454-9583-3969b21e4c26`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:56:03,465::task::595::Storage.TaskManager.Task::(_updateState) Task=`48df1d66-6ebf-4454-9583-3969b21e4c26`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 06:56:03,465::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 06:56:03,465::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 06:56:03,465::task::993::Storage.TaskManager.Task::(_decref) Task=`48df1d66-6ebf-4454-9583-3969b21e4c26`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:56:04,263::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:05,198::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:05,199::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-495::DEBUG::2014-10-15 06:56:05,200::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:05,201::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:05,202::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-496::DEBUG::2014-10-15 06:56:05,202::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-496::DEBUG::2014-10-15 06:56:05,203::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:05,265::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:06,266::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:56:06,531::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:06,531::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:06,531::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:06,531::__init__::232::IOProcess::(_processLogs) (407) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:56:06,531::__init__::232::IOProcess::(_processLogs) (407) Queuing response >Thread-25::DEBUG::2014-10-15 06:56:06,532::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:56:06,539::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000462206 s, 993 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:56:06,539::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:06,539::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:06,539::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:06,540::__init__::232::IOProcess::(_processLogs) (408) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:56:06,540::__init__::232::IOProcess::(_processLogs) (408) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:06,540::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:06,540::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:06,540::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:06,540::__init__::232::IOProcess::(_processLogs) (409) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:06,541::__init__::232::IOProcess::(_processLogs) (409) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:06,541::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:06,541::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:06,541::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:06,541::__init__::232::IOProcess::(_processLogs) (410) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:06,541::__init__::232::IOProcess::(_processLogs) (410) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:06,542::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:06,542::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:06,542::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:06,542::__init__::232::IOProcess::(_processLogs) (411) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:06,542::__init__::232::IOProcess::(_processLogs) (411) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:56:07,267::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:08,221::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:08,222::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-497::DEBUG::2014-10-15 06:56:08,223::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:08,224::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:08,225::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-498::DEBUG::2014-10-15 06:56:08,226::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-498::DEBUG::2014-10-15 06:56:08,226::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:08,268::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:08,269::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:56:08,269::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:08,307::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:08,308::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:08,308::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:56:09,270::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:10,271::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:11,254::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:11,254::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-499::DEBUG::2014-10-15 06:56:11,256::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2feaf9a-6a97-41d2-8e4c-73184d9bdf73`::moving from state init -> state preparing >Thread-499::INFO::2014-10-15 06:56:11,256::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-499::INFO::2014-10-15 06:56:11,256::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000462206', 'lastCheck': '4.7', 'valid': True}} >Thread-499::DEBUG::2014-10-15 06:56:11,256::task::1191::Storage.TaskManager.Task::(prepare) Task=`e2feaf9a-6a97-41d2-8e4c-73184d9bdf73`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000462206', 'lastCheck': '4.7', 'valid': True}} >Thread-499::DEBUG::2014-10-15 06:56:11,256::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2feaf9a-6a97-41d2-8e4c-73184d9bdf73`::moving from state preparing -> state finished >Thread-499::DEBUG::2014-10-15 06:56:11,256::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-499::DEBUG::2014-10-15 06:56:11,257::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-499::DEBUG::2014-10-15 06:56:11,257::task::993::Storage.TaskManager.Task::(_decref) Task=`e2feaf9a-6a97-41d2-8e4c-73184d9bdf73`::ref 0 aborting False >Thread-499::ERROR::2014-10-15 06:56:11,257::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-499::DEBUG::2014-10-15 06:56:11,258::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:11,269::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:11,269::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-500::DEBUG::2014-10-15 06:56:11,270::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-500::DEBUG::2014-10-15 06:56:11,271::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:11,272::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:12,274::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:13,275::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:13,314::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:13,314::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:13,315::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:56:14,276::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:14,387::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:14,388::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-501::DEBUG::2014-10-15 06:56:14,389::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:14,390::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:14,390::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-502::DEBUG::2014-10-15 06:56:14,391::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-502::DEBUG::2014-10-15 06:56:14,392::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:15,277::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:16,279::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:56:16,543::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:16,543::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:16,543::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:16,544::__init__::232::IOProcess::(_processLogs) (412) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:56:16,544::__init__::232::IOProcess::(_processLogs) (412) Queuing response >Thread-25::DEBUG::2014-10-15 06:56:16,544::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:56:16,550::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000362236 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:56:16,551::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:16,551::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:16,551::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:16,551::__init__::232::IOProcess::(_processLogs) (413) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:56:16,551::__init__::232::IOProcess::(_processLogs) (413) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:16,552::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:16,552::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:16,552::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:16,552::__init__::232::IOProcess::(_processLogs) (414) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:16,552::__init__::232::IOProcess::(_processLogs) (414) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:16,553::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:16,553::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:16,553::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:16,553::__init__::232::IOProcess::(_processLogs) (415) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:16,553::__init__::232::IOProcess::(_processLogs) (415) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:16,553::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:16,553::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:16,554::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:16,554::__init__::232::IOProcess::(_processLogs) (416) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:16,554::__init__::232::IOProcess::(_processLogs) (416) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:56:17,279::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:17,418::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:17,418::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-503::DEBUG::2014-10-15 06:56:17,419::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:17,420::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:17,421::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-504::DEBUG::2014-10-15 06:56:17,422::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-504::DEBUG::2014-10-15 06:56:17,422::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:18,280::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:18,316::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:18,316::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:18,317::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:56:19,282::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:20,283::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:20,438::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:20,439::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-505::DEBUG::2014-10-15 06:56:20,440::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:20,441::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:20,441::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-506::DEBUG::2014-10-15 06:56:20,442::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-506::DEBUG::2014-10-15 06:56:20,443::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:21,284::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:22,286::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:23,287::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:23,323::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:23,323::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:23,323::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:23,460::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:23,460::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-507::DEBUG::2014-10-15 06:56:23,461::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:23,462::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:23,463::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-508::DEBUG::2014-10-15 06:56:23,464::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-508::DEBUG::2014-10-15 06:56:23,464::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:24,288::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:25,289::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:26,291::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:26,490::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:26,491::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-509::DEBUG::2014-10-15 06:56:26,492::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f2da9f8-8ba4-42a1-ad1f-9cb2a2728540`::moving from state init -> state preparing >Thread-509::INFO::2014-10-15 06:56:26,493::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-509::INFO::2014-10-15 06:56:26,493::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000362236', 'lastCheck': '9.9', 'valid': True}} >Thread-509::DEBUG::2014-10-15 06:56:26,493::task::1191::Storage.TaskManager.Task::(prepare) Task=`4f2da9f8-8ba4-42a1-ad1f-9cb2a2728540`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000362236', 'lastCheck': '9.9', 'valid': True}} >Thread-509::DEBUG::2014-10-15 06:56:26,493::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f2da9f8-8ba4-42a1-ad1f-9cb2a2728540`::moving from state preparing -> state finished >Thread-509::DEBUG::2014-10-15 06:56:26,493::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-509::DEBUG::2014-10-15 06:56:26,493::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-509::DEBUG::2014-10-15 06:56:26,493::task::993::Storage.TaskManager.Task::(_decref) Task=`4f2da9f8-8ba4-42a1-ad1f-9cb2a2728540`::ref 0 aborting False >Thread-509::ERROR::2014-10-15 06:56:26,494::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-509::DEBUG::2014-10-15 06:56:26,495::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:26,505::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:26,505::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-510::DEBUG::2014-10-15 06:56:26,506::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-510::DEBUG::2014-10-15 06:56:26,507::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 06:56:26,555::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:26,555::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:26,555::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:26,555::__init__::232::IOProcess::(_processLogs) (417) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:56:26,555::__init__::232::IOProcess::(_processLogs) (417) Queuing response >Thread-25::DEBUG::2014-10-15 06:56:26,555::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:56:26,561::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000409826 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:56:26,562::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:26,562::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:26,562::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:26,563::__init__::232::IOProcess::(_processLogs) (418) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:56:26,563::__init__::232::IOProcess::(_processLogs) (418) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:26,563::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:26,563::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:26,563::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:26,563::__init__::232::IOProcess::(_processLogs) (419) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:26,564::__init__::232::IOProcess::(_processLogs) (419) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:26,564::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:26,564::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:26,564::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:26,564::__init__::232::IOProcess::(_processLogs) (420) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:26,564::__init__::232::IOProcess::(_processLogs) (420) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:26,565::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:26,565::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:26,565::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:26,565::__init__::232::IOProcess::(_processLogs) (421) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:26,565::__init__::232::IOProcess::(_processLogs) (421) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:56:27,293::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:28,294::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:28,326::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:28,326::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:28,326::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:56:29,295::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:29,628::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:29,629::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-511::DEBUG::2014-10-15 06:56:29,630::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:29,631::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:29,632::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-512::DEBUG::2014-10-15 06:56:29,632::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-512::DEBUG::2014-10-15 06:56:29,633::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:30,296::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:31,298::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:32,299::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:32,705::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:32,706::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-513::DEBUG::2014-10-15 06:56:32,707::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:32,708::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:32,709::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-514::DEBUG::2014-10-15 06:56:32,709::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-514::DEBUG::2014-10-15 06:56:32,710::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:33,301::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:33,332::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:33,333::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:33,333::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:56:34,302::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:35,303::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:35,729::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:35,730::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-515::DEBUG::2014-10-15 06:56:35,731::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:35,731::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:35,732::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-516::DEBUG::2014-10-15 06:56:35,733::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-516::DEBUG::2014-10-15 06:56:35,733::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:36,304::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:56:36,566::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:36,566::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:36,566::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:36,566::__init__::232::IOProcess::(_processLogs) (422) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:56:36,566::__init__::232::IOProcess::(_processLogs) (422) Queuing response >Thread-25::DEBUG::2014-10-15 06:56:36,567::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:56:36,573::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000394175 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:56:36,573::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:36,574::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:36,574::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:36,574::__init__::232::IOProcess::(_processLogs) (423) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:56:36,574::__init__::232::IOProcess::(_processLogs) (423) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:36,574::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:36,575::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:36,575::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:36,575::__init__::232::IOProcess::(_processLogs) (424) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:36,575::__init__::232::IOProcess::(_processLogs) (424) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:36,575::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:36,575::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:36,575::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:36,576::__init__::232::IOProcess::(_processLogs) (425) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:36,576::__init__::232::IOProcess::(_processLogs) (425) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:36,576::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:36,576::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:36,576::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:36,576::__init__::232::IOProcess::(_processLogs) (426) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:36,576::__init__::232::IOProcess::(_processLogs) (426) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:56:37,306::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:38,307::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:38,307::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:56:38,307::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:38,338::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:38,338::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:38,339::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:38,756::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:38,756::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-517::DEBUG::2014-10-15 06:56:38,757::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:38,758::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:38,759::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-518::DEBUG::2014-10-15 06:56:38,760::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-518::DEBUG::2014-10-15 06:56:38,760::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:39,309::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:40,310::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:41,311::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:41,783::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:41,784::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-519::DEBUG::2014-10-15 06:56:41,785::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e3d29a7-1de2-426b-b8f7-555c982c43f5`::moving from state init -> state preparing >Thread-519::INFO::2014-10-15 06:56:41,786::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-519::INFO::2014-10-15 06:56:41,786::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000394175', 'lastCheck': '5.2', 'valid': True}} >Thread-519::DEBUG::2014-10-15 06:56:41,786::task::1191::Storage.TaskManager.Task::(prepare) Task=`9e3d29a7-1de2-426b-b8f7-555c982c43f5`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000394175', 'lastCheck': '5.2', 'valid': True}} >Thread-519::DEBUG::2014-10-15 06:56:41,786::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e3d29a7-1de2-426b-b8f7-555c982c43f5`::moving from state preparing -> state finished >Thread-519::DEBUG::2014-10-15 06:56:41,786::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-519::DEBUG::2014-10-15 06:56:41,786::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-519::DEBUG::2014-10-15 06:56:41,786::task::993::Storage.TaskManager.Task::(_decref) Task=`9e3d29a7-1de2-426b-b8f7-555c982c43f5`::ref 0 aborting False >Thread-519::ERROR::2014-10-15 06:56:41,787::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-519::DEBUG::2014-10-15 06:56:41,788::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:41,800::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:41,800::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-520::DEBUG::2014-10-15 06:56:41,801::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-520::DEBUG::2014-10-15 06:56:41,802::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:42,315::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:43,316::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:43,344::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:43,345::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:43,345::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:56:44,317::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:44,910::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:44,911::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-521::DEBUG::2014-10-15 06:56:44,911::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:44,913::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:44,913::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-522::DEBUG::2014-10-15 06:56:44,914::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-522::DEBUG::2014-10-15 06:56:44,914::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:45,318::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:46,320::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:56:46,577::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:46,578::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:46,578::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:46,578::__init__::232::IOProcess::(_processLogs) (427) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:56:46,578::__init__::232::IOProcess::(_processLogs) (427) Queuing response >Thread-25::DEBUG::2014-10-15 06:56:46,579::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:56:46,585::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000436501 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:56:46,585::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:46,586::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:46,586::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:46,586::__init__::232::IOProcess::(_processLogs) (428) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:56:46,586::__init__::232::IOProcess::(_processLogs) (428) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:46,586::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:46,587::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:46,587::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:46,587::__init__::232::IOProcess::(_processLogs) (429) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:46,587::__init__::232::IOProcess::(_processLogs) (429) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:46,587::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:46,587::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:46,587::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:46,588::__init__::232::IOProcess::(_processLogs) (430) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:46,588::__init__::232::IOProcess::(_processLogs) (430) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:46,588::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:46,588::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:46,588::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:46,588::__init__::232::IOProcess::(_processLogs) (431) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:46,588::__init__::232::IOProcess::(_processLogs) (431) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:56:47,321::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:47,952::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:47,953::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-523::DEBUG::2014-10-15 06:56:47,954::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:47,955::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:47,956::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-524::DEBUG::2014-10-15 06:56:47,957::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-524::DEBUG::2014-10-15 06:56:47,957::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:48,322::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:48,351::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:48,351::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:48,352::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:56:49,324::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:50,325::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:50,980::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:50,980::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-525::DEBUG::2014-10-15 06:56:50,981::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:50,982::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:50,983::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-526::DEBUG::2014-10-15 06:56:50,984::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-526::DEBUG::2014-10-15 06:56:50,984::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:51,326::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:52,328::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:53,329::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:53,354::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:53,355::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:53,355::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:54,000::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:54,001::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-527::DEBUG::2014-10-15 06:56:54,002::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:54,003::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:54,004::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-528::DEBUG::2014-10-15 06:56:54,004::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-528::DEBUG::2014-10-15 06:56:54,005::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:54,330::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:55,331::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:56,332::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:56:56,590::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:56,590::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:56,590::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:56,590::__init__::232::IOProcess::(_processLogs) (432) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:56:56,590::__init__::232::IOProcess::(_processLogs) (432) Queuing response >Thread-25::DEBUG::2014-10-15 06:56:56,591::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:56:56,596::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000464679 s, 988 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:56:56,597::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:56,597::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:56,597::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:56,597::__init__::232::IOProcess::(_processLogs) (433) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:56:56,598::__init__::232::IOProcess::(_processLogs) (433) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:56,598::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:56,598::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:56,598::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:56,598::__init__::232::IOProcess::(_processLogs) (434) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:56,599::__init__::232::IOProcess::(_processLogs) (434) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:56,599::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:56,599::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:56,599::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:56,599::__init__::232::IOProcess::(_processLogs) (435) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:56,599::__init__::232::IOProcess::(_processLogs) (435) Queuing response >Thread-16::DEBUG::2014-10-15 06:56:56,600::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:56:56,600::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:56:56,600::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:56:56,600::__init__::232::IOProcess::(_processLogs) (436) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:56:56,600::__init__::232::IOProcess::(_processLogs) (436) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:57,022::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:57,022::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-529::DEBUG::2014-10-15 06:56:57,024::task::595::Storage.TaskManager.Task::(_updateState) Task=`f9414065-fef8-46fe-a979-6d529d41f535`::moving from state init -> state preparing >Thread-529::INFO::2014-10-15 06:56:57,024::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-529::INFO::2014-10-15 06:56:57,024::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000464679', 'lastCheck': '0.4', 'valid': True}} >Thread-529::DEBUG::2014-10-15 06:56:57,024::task::1191::Storage.TaskManager.Task::(prepare) Task=`f9414065-fef8-46fe-a979-6d529d41f535`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000464679', 'lastCheck': '0.4', 'valid': True}} >Thread-529::DEBUG::2014-10-15 06:56:57,024::task::595::Storage.TaskManager.Task::(_updateState) Task=`f9414065-fef8-46fe-a979-6d529d41f535`::moving from state preparing -> state finished >Thread-529::DEBUG::2014-10-15 06:56:57,024::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-529::DEBUG::2014-10-15 06:56:57,025::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-529::DEBUG::2014-10-15 06:56:57,025::task::993::Storage.TaskManager.Task::(_decref) Task=`f9414065-fef8-46fe-a979-6d529d41f535`::ref 0 aborting False >Thread-529::ERROR::2014-10-15 06:56:57,026::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-529::DEBUG::2014-10-15 06:56:57,027::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:56:57,039::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:56:57,039::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-530::DEBUG::2014-10-15 06:56:57,041::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-530::DEBUG::2014-10-15 06:56:57,041::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:56:57,334::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:56:58,335::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:58,360::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:58,361::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:56:58,361::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:56:59,336::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:00,150::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:00,150::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-531::DEBUG::2014-10-15 06:57:00,151::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:00,153::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:00,153::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-532::DEBUG::2014-10-15 06:57:00,154::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-532::DEBUG::2014-10-15 06:57:00,154::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:00,337::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:01,339::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:02,340::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:03,191::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:03,192::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-533::DEBUG::2014-10-15 06:57:03,193::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:03,194::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:03,194::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-534::DEBUG::2014-10-15 06:57:03,195::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-534::DEBUG::2014-10-15 06:57:03,196::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:03,342::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:03,365::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:03,365::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:03,366::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 06:57:03,507::task::595::Storage.TaskManager.Task::(_updateState) Task=`151a30c1-aef7-4194-b97b-72f481d54c6a`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 06:57:03,507::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:57:03,508::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:03,508::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:03,508::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:03,508::__init__::232::IOProcess::(_processLogs) (437) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:57:03,509::__init__::232::IOProcess::(_processLogs) (437) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:03,510::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:03,510::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:03,510::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:03,510::__init__::232::IOProcess::(_processLogs) (438) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:57:03,510::__init__::232::IOProcess::(_processLogs) (438) Queuing response >Thread-17::INFO::2014-10-15 06:57:03,510::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:57:03,511::task::1191::Storage.TaskManager.Task::(prepare) Task=`151a30c1-aef7-4194-b97b-72f481d54c6a`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:57:03,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`151a30c1-aef7-4194-b97b-72f481d54c6a`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 06:57:03,511::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 06:57:03,511::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 06:57:03,511::task::993::Storage.TaskManager.Task::(_decref) Task=`151a30c1-aef7-4194-b97b-72f481d54c6a`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:57:04,343::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:05,344::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:06,211::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:06,211::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-535::DEBUG::2014-10-15 06:57:06,212::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:06,213::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:06,214::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-536::DEBUG::2014-10-15 06:57:06,215::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-536::DEBUG::2014-10-15 06:57:06,215::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:06,346::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:57:06,601::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:06,601::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:06,602::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:06,602::__init__::232::IOProcess::(_processLogs) (439) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:57:06,602::__init__::232::IOProcess::(_processLogs) (439) Queuing response >Thread-25::DEBUG::2014-10-15 06:57:06,602::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:57:06,609::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000360344 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:57:06,610::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:06,610::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:06,610::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:06,610::__init__::232::IOProcess::(_processLogs) (440) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:57:06,610::__init__::232::IOProcess::(_processLogs) (440) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:06,611::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:06,611::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:06,611::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:06,611::__init__::232::IOProcess::(_processLogs) (441) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:06,611::__init__::232::IOProcess::(_processLogs) (441) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:06,612::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:06,612::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:06,612::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:06,612::__init__::232::IOProcess::(_processLogs) (442) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:06,612::__init__::232::IOProcess::(_processLogs) (442) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:06,612::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:06,612::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:06,612::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:06,613::__init__::232::IOProcess::(_processLogs) (443) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:06,613::__init__::232::IOProcess::(_processLogs) (443) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:57:07,347::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:08,348::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:08,348::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:57:08,349::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:08,371::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:08,372::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:08,372::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:09,230::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:09,231::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-537::DEBUG::2014-10-15 06:57:09,232::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:09,233::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:09,234::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-538::DEBUG::2014-10-15 06:57:09,235::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-538::DEBUG::2014-10-15 06:57:09,235::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:09,350::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:10,351::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:11,352::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:12,257::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:12,257::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-539::DEBUG::2014-10-15 06:57:12,259::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb40f9ba-46d3-4c5b-84aa-5560f3fc6952`::moving from state init -> state preparing >Thread-539::INFO::2014-10-15 06:57:12,259::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-539::INFO::2014-10-15 06:57:12,259::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000360344', 'lastCheck': '5.6', 'valid': True}} >Thread-539::DEBUG::2014-10-15 06:57:12,259::task::1191::Storage.TaskManager.Task::(prepare) Task=`eb40f9ba-46d3-4c5b-84aa-5560f3fc6952`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000360344', 'lastCheck': '5.6', 'valid': True}} >Thread-539::DEBUG::2014-10-15 06:57:12,259::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb40f9ba-46d3-4c5b-84aa-5560f3fc6952`::moving from state preparing -> state finished >Thread-539::DEBUG::2014-10-15 06:57:12,259::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-539::DEBUG::2014-10-15 06:57:12,259::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-539::DEBUG::2014-10-15 06:57:12,260::task::993::Storage.TaskManager.Task::(_decref) Task=`eb40f9ba-46d3-4c5b-84aa-5560f3fc6952`::ref 0 aborting False >Thread-539::ERROR::2014-10-15 06:57:12,260::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-539::DEBUG::2014-10-15 06:57:12,261::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:12,272::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:12,273::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-540::DEBUG::2014-10-15 06:57:12,274::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-540::DEBUG::2014-10-15 06:57:12,275::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:12,354::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:13,355::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:13,378::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:13,378::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:13,379::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:57:14,356::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:15,356::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:15,396::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:15,396::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-541::DEBUG::2014-10-15 06:57:15,397::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:15,399::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:15,399::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-542::DEBUG::2014-10-15 06:57:15,400::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-542::DEBUG::2014-10-15 06:57:15,400::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:16,358::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:57:16,614::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:16,614::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:16,614::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:16,614::__init__::232::IOProcess::(_processLogs) (444) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:57:16,614::__init__::232::IOProcess::(_processLogs) (444) Queuing response >Thread-25::DEBUG::2014-10-15 06:57:16,615::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:57:16,621::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000395913 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:57:16,622::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:16,622::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:16,622::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:16,622::__init__::232::IOProcess::(_processLogs) (445) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:57:16,622::__init__::232::IOProcess::(_processLogs) (445) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:16,623::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:16,623::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:16,623::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:16,623::__init__::232::IOProcess::(_processLogs) (446) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:16,623::__init__::232::IOProcess::(_processLogs) (446) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:16,624::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:16,624::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:16,624::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:16,624::__init__::232::IOProcess::(_processLogs) (447) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:16,624::__init__::232::IOProcess::(_processLogs) (447) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:16,624::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:16,624::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:16,624::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:16,625::__init__::232::IOProcess::(_processLogs) (448) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:16,625::__init__::232::IOProcess::(_processLogs) (448) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:57:17,359::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:18,360::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:18,382::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:18,383::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:18,383::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:18,558::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:18,559::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-543::DEBUG::2014-10-15 06:57:18,560::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:18,561::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:18,562::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-544::DEBUG::2014-10-15 06:57:18,562::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-544::DEBUG::2014-10-15 06:57:18,563::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:19,362::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:20,363::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:21,364::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:21,584::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:21,585::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-545::DEBUG::2014-10-15 06:57:21,586::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:21,587::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:21,588::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-546::DEBUG::2014-10-15 06:57:21,589::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-546::DEBUG::2014-10-15 06:57:21,589::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:22,365::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:23,367::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:23,386::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:23,386::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:23,387::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:57:24,368::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:24,612::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:24,613::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-547::DEBUG::2014-10-15 06:57:24,614::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:24,615::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:24,615::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-548::DEBUG::2014-10-15 06:57:24,616::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-548::DEBUG::2014-10-15 06:57:24,616::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:25,369::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:26,370::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:57:26,627::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:26,627::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:26,627::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:26,627::__init__::232::IOProcess::(_processLogs) (449) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:57:26,627::__init__::232::IOProcess::(_processLogs) (449) Queuing response >Thread-25::DEBUG::2014-10-15 06:57:26,627::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:57:26,634::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000396649 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:57:26,634::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:26,634::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:26,634::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:26,635::__init__::232::IOProcess::(_processLogs) (450) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:57:26,635::__init__::232::IOProcess::(_processLogs) (450) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:26,635::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:26,635::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:26,636::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:26,636::__init__::232::IOProcess::(_processLogs) (451) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:26,636::__init__::232::IOProcess::(_processLogs) (451) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:26,636::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:26,636::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:26,636::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:26,636::__init__::232::IOProcess::(_processLogs) (452) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:26,636::__init__::232::IOProcess::(_processLogs) (452) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:26,637::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:26,637::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:26,637::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:26,637::__init__::232::IOProcess::(_processLogs) (453) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:26,637::__init__::232::IOProcess::(_processLogs) (453) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:57:27,372::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:27,641::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:27,642::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-549::DEBUG::2014-10-15 06:57:27,643::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9853e54-fded-44e0-a391-12cbf994fcca`::moving from state init -> state preparing >Thread-549::INFO::2014-10-15 06:57:27,643::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-549::INFO::2014-10-15 06:57:27,643::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000396649', 'lastCheck': '1.0', 'valid': True}} >Thread-549::DEBUG::2014-10-15 06:57:27,643::task::1191::Storage.TaskManager.Task::(prepare) Task=`d9853e54-fded-44e0-a391-12cbf994fcca`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000396649', 'lastCheck': '1.0', 'valid': True}} >Thread-549::DEBUG::2014-10-15 06:57:27,644::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9853e54-fded-44e0-a391-12cbf994fcca`::moving from state preparing -> state finished >Thread-549::DEBUG::2014-10-15 06:57:27,644::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-549::DEBUG::2014-10-15 06:57:27,644::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-549::DEBUG::2014-10-15 06:57:27,644::task::993::Storage.TaskManager.Task::(_decref) Task=`d9853e54-fded-44e0-a391-12cbf994fcca`::ref 0 aborting False >Thread-549::ERROR::2014-10-15 06:57:27,645::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-549::DEBUG::2014-10-15 06:57:27,646::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:27,655::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:27,656::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-550::DEBUG::2014-10-15 06:57:27,657::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-550::DEBUG::2014-10-15 06:57:27,657::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:28,373::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:28,393::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:28,393::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:28,393::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:57:29,374::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:30,375::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:30,774::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:30,775::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-551::DEBUG::2014-10-15 06:57:30,776::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:30,777::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:30,777::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-552::DEBUG::2014-10-15 06:57:30,778::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-552::DEBUG::2014-10-15 06:57:30,778::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:31,376::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:32,377::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:33,378::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:33,399::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:33,400::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:33,400::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:33,832::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:33,833::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-553::DEBUG::2014-10-15 06:57:33,834::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:33,835::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:33,836::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-554::DEBUG::2014-10-15 06:57:33,837::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-554::DEBUG::2014-10-15 06:57:33,837::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:34,379::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:35,380::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:36,381::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:57:36,638::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:36,639::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:36,639::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:36,639::__init__::232::IOProcess::(_processLogs) (454) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:57:36,639::__init__::232::IOProcess::(_processLogs) (454) Queuing response >Thread-25::DEBUG::2014-10-15 06:57:36,639::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:57:36,645::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000435066 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:57:36,646::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:36,646::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:36,646::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:36,647::__init__::232::IOProcess::(_processLogs) (455) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:57:36,647::__init__::232::IOProcess::(_processLogs) (455) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:36,647::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:36,647::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:36,647::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:36,648::__init__::232::IOProcess::(_processLogs) (456) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:36,648::__init__::232::IOProcess::(_processLogs) (456) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:36,648::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:36,648::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:36,648::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:36,648::__init__::232::IOProcess::(_processLogs) (457) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:36,649::__init__::232::IOProcess::(_processLogs) (457) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:36,649::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:36,649::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:36,649::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:36,649::__init__::232::IOProcess::(_processLogs) (458) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:36,649::__init__::232::IOProcess::(_processLogs) (458) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:36,856::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:36,857::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-555::DEBUG::2014-10-15 06:57:36,858::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:36,859::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:36,860::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-556::DEBUG::2014-10-15 06:57:36,861::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-556::DEBUG::2014-10-15 06:57:36,861::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:37,382::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:38,384::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:38,384::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:57:38,384::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:38,406::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:38,407::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:38,407::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:57:39,386::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:39,883::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:39,884::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-557::DEBUG::2014-10-15 06:57:39,885::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:39,886::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:39,887::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-558::DEBUG::2014-10-15 06:57:39,888::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-558::DEBUG::2014-10-15 06:57:39,888::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:40,387::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:41,388::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:42,389::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:42,916::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:42,916::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-559::DEBUG::2014-10-15 06:57:42,918::task::595::Storage.TaskManager.Task::(_updateState) Task=`61e3d064-e826-4631-93d0-88119e840300`::moving from state init -> state preparing >Thread-559::INFO::2014-10-15 06:57:42,918::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-559::INFO::2014-10-15 06:57:42,919::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000435066', 'lastCheck': '6.3', 'valid': True}} >Thread-559::DEBUG::2014-10-15 06:57:42,919::task::1191::Storage.TaskManager.Task::(prepare) Task=`61e3d064-e826-4631-93d0-88119e840300`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000435066', 'lastCheck': '6.3', 'valid': True}} >Thread-559::DEBUG::2014-10-15 06:57:42,919::task::595::Storage.TaskManager.Task::(_updateState) Task=`61e3d064-e826-4631-93d0-88119e840300`::moving from state preparing -> state finished >Thread-559::DEBUG::2014-10-15 06:57:42,919::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-559::DEBUG::2014-10-15 06:57:42,919::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-559::DEBUG::2014-10-15 06:57:42,919::task::993::Storage.TaskManager.Task::(_decref) Task=`61e3d064-e826-4631-93d0-88119e840300`::ref 0 aborting False >Thread-559::ERROR::2014-10-15 06:57:42,920::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-559::DEBUG::2014-10-15 06:57:42,921::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:42,932::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:42,933::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-560::DEBUG::2014-10-15 06:57:42,934::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-560::DEBUG::2014-10-15 06:57:42,934::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:43,391::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:43,413::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:43,413::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:43,413::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:57:44,392::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:45,393::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:46,063::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:46,064::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-561::DEBUG::2014-10-15 06:57:46,064::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:46,066::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:46,066::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-562::DEBUG::2014-10-15 06:57:46,067::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-562::DEBUG::2014-10-15 06:57:46,067::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:46,394::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:57:46,652::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:46,652::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:46,654::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:46,654::__init__::232::IOProcess::(_processLogs) (459) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:57:46,655::__init__::232::IOProcess::(_processLogs) (459) Queuing response >Thread-25::DEBUG::2014-10-15 06:57:46,655::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:57:46,662::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000384011 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:57:46,663::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:46,663::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:46,663::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:46,665::__init__::232::IOProcess::(_processLogs) (460) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:57:46,665::__init__::232::IOProcess::(_processLogs) (460) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:46,666::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:46,667::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:46,667::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:46,667::__init__::232::IOProcess::(_processLogs) (461) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:46,667::__init__::232::IOProcess::(_processLogs) (461) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:46,667::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:46,667::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:46,667::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:46,667::__init__::232::IOProcess::(_processLogs) (462) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:46,668::__init__::232::IOProcess::(_processLogs) (462) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:46,668::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:46,668::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:46,668::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:46,668::__init__::232::IOProcess::(_processLogs) (463) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:46,668::__init__::232::IOProcess::(_processLogs) (463) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:57:47,396::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:48,397::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:48,419::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:48,420::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:48,420::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:49,105::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:49,106::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-563::DEBUG::2014-10-15 06:57:49,106::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:49,108::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:49,108::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-564::DEBUG::2014-10-15 06:57:49,109::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-564::DEBUG::2014-10-15 06:57:49,109::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:49,398::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:50,399::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:51,401::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:52,131::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:52,132::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-565::DEBUG::2014-10-15 06:57:52,133::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:52,134::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:52,135::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-566::DEBUG::2014-10-15 06:57:52,136::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-566::DEBUG::2014-10-15 06:57:52,136::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:52,402::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:53,403::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:53,425::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:53,425::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:53,426::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:57:54,404::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:55,160::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:55,161::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-567::DEBUG::2014-10-15 06:57:55,161::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:55,163::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:55,163::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-568::DEBUG::2014-10-15 06:57:55,164::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-568::DEBUG::2014-10-15 06:57:55,164::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:55,406::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:57:56,407::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:57:56,669::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:56,669::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:56,669::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:56,670::__init__::232::IOProcess::(_processLogs) (464) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:57:56,670::__init__::232::IOProcess::(_processLogs) (464) Queuing response >Thread-25::DEBUG::2014-10-15 06:57:56,670::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:57:56,676::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000354012 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:57:56,676::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:56,677::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:56,677::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:56,677::__init__::232::IOProcess::(_processLogs) (465) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:57:56,677::__init__::232::IOProcess::(_processLogs) (465) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:56,678::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:56,678::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:56,678::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:56,678::__init__::232::IOProcess::(_processLogs) (466) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:56,678::__init__::232::IOProcess::(_processLogs) (466) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:56,678::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:56,679::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:56,679::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:56,679::__init__::232::IOProcess::(_processLogs) (467) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:56,679::__init__::232::IOProcess::(_processLogs) (467) Queuing response >Thread-16::DEBUG::2014-10-15 06:57:56,679::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:57:56,679::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:57:56,679::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:57:56,679::__init__::232::IOProcess::(_processLogs) (468) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:57:56,680::__init__::232::IOProcess::(_processLogs) (468) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:57:57,408::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:58,185::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:58,186::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-569::DEBUG::2014-10-15 06:57:58,187::task::595::Storage.TaskManager.Task::(_updateState) Task=`500c6207-246e-4784-a501-f08b623af72d`::moving from state init -> state preparing >Thread-569::INFO::2014-10-15 06:57:58,187::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-569::INFO::2014-10-15 06:57:58,187::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000354012', 'lastCheck': '1.5', 'valid': True}} >Thread-569::DEBUG::2014-10-15 06:57:58,188::task::1191::Storage.TaskManager.Task::(prepare) Task=`500c6207-246e-4784-a501-f08b623af72d`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000354012', 'lastCheck': '1.5', 'valid': True}} >Thread-569::DEBUG::2014-10-15 06:57:58,188::task::595::Storage.TaskManager.Task::(_updateState) Task=`500c6207-246e-4784-a501-f08b623af72d`::moving from state preparing -> state finished >Thread-569::DEBUG::2014-10-15 06:57:58,188::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-569::DEBUG::2014-10-15 06:57:58,188::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-569::DEBUG::2014-10-15 06:57:58,188::task::993::Storage.TaskManager.Task::(_decref) Task=`500c6207-246e-4784-a501-f08b623af72d`::ref 0 aborting False >Thread-569::ERROR::2014-10-15 06:57:58,189::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-569::DEBUG::2014-10-15 06:57:58,189::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:57:58,202::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:57:58,202::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-570::DEBUG::2014-10-15 06:57:58,203::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-570::DEBUG::2014-10-15 06:57:58,204::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:57:58,409::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:58,431::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:58,432::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:57:58,432::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:57:59,411::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:00,412::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:01,312::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:01,313::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-571::DEBUG::2014-10-15 06:58:01,313::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:01,315::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:01,315::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-572::DEBUG::2014-10-15 06:58:01,316::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-572::DEBUG::2014-10-15 06:58:01,316::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:01,413::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:02,414::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:03,416::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:03,438::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:03,438::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:03,439::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 06:58:03,556::task::595::Storage.TaskManager.Task::(_updateState) Task=`0111c122-1443-4e9c-a93a-36e5169a5d87`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 06:58:03,556::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:58:03,557::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:03,557::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:03,557::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:03,557::__init__::232::IOProcess::(_processLogs) (469) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:58:03,559::__init__::232::IOProcess::(_processLogs) (469) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:03,559::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:03,559::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:03,559::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:03,559::__init__::232::IOProcess::(_processLogs) (470) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:58:03,560::__init__::232::IOProcess::(_processLogs) (470) Queuing response >Thread-17::INFO::2014-10-15 06:58:03,560::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:58:03,560::task::1191::Storage.TaskManager.Task::(prepare) Task=`0111c122-1443-4e9c-a93a-36e5169a5d87`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:58:03,560::task::595::Storage.TaskManager.Task::(_updateState) Task=`0111c122-1443-4e9c-a93a-36e5169a5d87`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 06:58:03,560::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 06:58:03,560::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 06:58:03,560::task::993::Storage.TaskManager.Task::(_decref) Task=`0111c122-1443-4e9c-a93a-36e5169a5d87`::ref 0 aborting False >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:04,345::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:04,346::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-573::DEBUG::2014-10-15 06:58:04,347::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:04,348::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:04,349::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-574::DEBUG::2014-10-15 06:58:04,350::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-574::DEBUG::2014-10-15 06:58:04,350::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:04,417::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:05,419::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:06,420::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:58:06,680::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:06,681::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:06,681::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:06,681::__init__::232::IOProcess::(_processLogs) (471) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:58:06,681::__init__::232::IOProcess::(_processLogs) (471) Queuing response >Thread-25::DEBUG::2014-10-15 06:58:06,681::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:58:06,688::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00040341 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:58:06,689::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:06,689::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:06,689::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:06,689::__init__::232::IOProcess::(_processLogs) (472) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:58:06,689::__init__::232::IOProcess::(_processLogs) (472) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:06,690::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:06,690::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:06,690::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:06,690::__init__::232::IOProcess::(_processLogs) (473) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:06,690::__init__::232::IOProcess::(_processLogs) (473) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:06,691::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:06,691::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:06,691::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:06,691::__init__::232::IOProcess::(_processLogs) (474) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:06,691::__init__::232::IOProcess::(_processLogs) (474) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:06,691::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:06,691::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:06,691::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:06,692::__init__::232::IOProcess::(_processLogs) (475) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:06,692::__init__::232::IOProcess::(_processLogs) (475) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:07,373::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:07,373::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-575::DEBUG::2014-10-15 06:58:07,375::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:07,376::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:07,376::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-576::DEBUG::2014-10-15 06:58:07,377::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-576::DEBUG::2014-10-15 06:58:07,378::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:07,421::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:08,422::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:08,422::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:58:08,423::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:08,445::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:08,445::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:08,445::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:58:09,424::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:10,397::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:10,398::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-577::DEBUG::2014-10-15 06:58:10,399::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:10,400::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:10,401::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-578::DEBUG::2014-10-15 06:58:10,402::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-578::DEBUG::2014-10-15 06:58:10,402::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:10,425::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:11,426::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:12,427::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:13,425::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:13,426::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-579::DEBUG::2014-10-15 06:58:13,427::task::595::Storage.TaskManager.Task::(_updateState) Task=`3aa2538b-47c1-42c4-ad3a-c7c2fc322d4c`::moving from state init -> state preparing >Thread-579::INFO::2014-10-15 06:58:13,427::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-579::INFO::2014-10-15 06:58:13,427::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00040341', 'lastCheck': '6.7', 'valid': True}} >Thread-579::DEBUG::2014-10-15 06:58:13,427::task::1191::Storage.TaskManager.Task::(prepare) Task=`3aa2538b-47c1-42c4-ad3a-c7c2fc322d4c`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00040341', 'lastCheck': '6.7', 'valid': True}} >Thread-579::DEBUG::2014-10-15 06:58:13,428::task::595::Storage.TaskManager.Task::(_updateState) Task=`3aa2538b-47c1-42c4-ad3a-c7c2fc322d4c`::moving from state preparing -> state finished >Thread-579::DEBUG::2014-10-15 06:58:13,428::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-579::DEBUG::2014-10-15 06:58:13,428::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-579::DEBUG::2014-10-15 06:58:13,428::task::993::Storage.TaskManager.Task::(_decref) Task=`3aa2538b-47c1-42c4-ad3a-c7c2fc322d4c`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:58:13,429::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-579::ERROR::2014-10-15 06:58:13,429::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-579::DEBUG::2014-10-15 06:58:13,430::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:13,439::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:13,440::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-580::DEBUG::2014-10-15 06:58:13,441::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-580::DEBUG::2014-10-15 06:58:13,441::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:13,451::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:13,451::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:13,452::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:58:14,431::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:15,432::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:16,433::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:16,542::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:16,543::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-581::DEBUG::2014-10-15 06:58:16,544::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:16,545::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:16,546::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-582::DEBUG::2014-10-15 06:58:16,546::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-582::DEBUG::2014-10-15 06:58:16,547::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 06:58:16,693::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:16,693::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:16,693::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:16,693::__init__::232::IOProcess::(_processLogs) (476) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:58:16,693::__init__::232::IOProcess::(_processLogs) (476) Queuing response >Thread-25::DEBUG::2014-10-15 06:58:16,694::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:58:16,700::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000446822 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:58:16,701::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:16,701::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:16,701::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:16,701::__init__::232::IOProcess::(_processLogs) (477) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:58:16,701::__init__::232::IOProcess::(_processLogs) (477) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:16,702::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:16,702::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:16,702::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:16,702::__init__::232::IOProcess::(_processLogs) (478) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:16,702::__init__::232::IOProcess::(_processLogs) (478) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:16,702::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:16,702::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:16,703::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:16,703::__init__::232::IOProcess::(_processLogs) (479) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:16,703::__init__::232::IOProcess::(_processLogs) (479) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:16,703::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:16,703::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:16,703::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:16,703::__init__::232::IOProcess::(_processLogs) (480) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:16,703::__init__::232::IOProcess::(_processLogs) (480) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:58:17,434::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:18,436::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:18,458::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:18,458::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:18,459::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:58:19,437::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:19,577::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:19,578::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-583::DEBUG::2014-10-15 06:58:19,579::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:19,580::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:19,580::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-584::DEBUG::2014-10-15 06:58:19,581::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-584::DEBUG::2014-10-15 06:58:19,582::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:20,438::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:21,440::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:22,441::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:22,598::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:22,599::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-585::DEBUG::2014-10-15 06:58:22,600::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:22,601::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:22,601::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-586::DEBUG::2014-10-15 06:58:22,602::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-586::DEBUG::2014-10-15 06:58:22,603::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:23,442::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:23,459::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:23,460::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:23,460::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:58:24,443::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:25,445::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:25,622::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:25,622::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-587::DEBUG::2014-10-15 06:58:25,623::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:25,624::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:25,625::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-588::DEBUG::2014-10-15 06:58:25,626::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-588::DEBUG::2014-10-15 06:58:25,626::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:26,446::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:58:26,704::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:26,704::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:26,705::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:26,705::__init__::232::IOProcess::(_processLogs) (481) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:58:26,705::__init__::232::IOProcess::(_processLogs) (481) Queuing response >Thread-25::DEBUG::2014-10-15 06:58:26,705::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:58:26,711::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000383598 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:58:26,712::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:26,712::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:26,712::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:26,712::__init__::232::IOProcess::(_processLogs) (482) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:58:26,713::__init__::232::IOProcess::(_processLogs) (482) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:26,713::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:26,713::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:26,713::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:26,713::__init__::232::IOProcess::(_processLogs) (483) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:26,713::__init__::232::IOProcess::(_processLogs) (483) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:26,714::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:26,714::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:26,714::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:26,714::__init__::232::IOProcess::(_processLogs) (484) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:26,714::__init__::232::IOProcess::(_processLogs) (484) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:26,714::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:26,715::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:26,715::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:26,715::__init__::232::IOProcess::(_processLogs) (485) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:26,715::__init__::232::IOProcess::(_processLogs) (485) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:58:27,447::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:28,448::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:28,466::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:28,466::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:28,467::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:28,665::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:28,666::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-589::DEBUG::2014-10-15 06:58:28,667::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7fbd2a9-f998-4e82-8d43-d54d7afb1390`::moving from state init -> state preparing >Thread-589::INFO::2014-10-15 06:58:28,667::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-589::INFO::2014-10-15 06:58:28,667::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000383598', 'lastCheck': '2.0', 'valid': True}} >Thread-589::DEBUG::2014-10-15 06:58:28,667::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7fbd2a9-f998-4e82-8d43-d54d7afb1390`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000383598', 'lastCheck': '2.0', 'valid': True}} >Thread-589::DEBUG::2014-10-15 06:58:28,667::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7fbd2a9-f998-4e82-8d43-d54d7afb1390`::moving from state preparing -> state finished >Thread-589::DEBUG::2014-10-15 06:58:28,668::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-589::DEBUG::2014-10-15 06:58:28,668::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-589::DEBUG::2014-10-15 06:58:28,668::task::993::Storage.TaskManager.Task::(_decref) Task=`c7fbd2a9-f998-4e82-8d43-d54d7afb1390`::ref 0 aborting False >Thread-589::ERROR::2014-10-15 06:58:28,669::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-589::DEBUG::2014-10-15 06:58:28,669::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:28,682::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:28,682::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-590::DEBUG::2014-10-15 06:58:28,683::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-590::DEBUG::2014-10-15 06:58:28,683::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:29,450::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:30,451::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:31,452::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:31,793::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:31,795::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-591::DEBUG::2014-10-15 06:58:31,797::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:31,799::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:31,800::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-592::DEBUG::2014-10-15 06:58:31,802::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-592::DEBUG::2014-10-15 06:58:31,804::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:32,454::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:33,455::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:33,473::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:33,473::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:33,474::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:58:34,456::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:34,845::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:34,846::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-593::DEBUG::2014-10-15 06:58:34,847::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:34,847::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:34,848::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-594::DEBUG::2014-10-15 06:58:34,849::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-594::DEBUG::2014-10-15 06:58:34,849::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:35,458::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:36,459::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:58:36,716::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:36,716::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:36,716::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:36,716::__init__::232::IOProcess::(_processLogs) (486) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:58:36,716::__init__::232::IOProcess::(_processLogs) (486) Queuing response >Thread-25::DEBUG::2014-10-15 06:58:36,717::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:58:36,723::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000402072 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:58:36,723::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:36,724::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:36,724::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:36,724::__init__::232::IOProcess::(_processLogs) (487) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:58:36,724::__init__::232::IOProcess::(_processLogs) (487) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:36,725::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:36,725::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:36,725::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:36,725::__init__::232::IOProcess::(_processLogs) (488) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:36,725::__init__::232::IOProcess::(_processLogs) (488) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:36,725::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:36,725::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:36,726::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:36,726::__init__::232::IOProcess::(_processLogs) (489) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:36,726::__init__::232::IOProcess::(_processLogs) (489) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:36,726::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:36,726::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:36,726::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:36,726::__init__::232::IOProcess::(_processLogs) (490) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:36,727::__init__::232::IOProcess::(_processLogs) (490) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:58:37,460::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:37,868::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:37,869::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-595::DEBUG::2014-10-15 06:58:37,870::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:37,871::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:37,872::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-596::DEBUG::2014-10-15 06:58:37,873::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-596::DEBUG::2014-10-15 06:58:37,873::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:38,461::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:38,462::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:58:38,462::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:38,479::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:38,480::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:38,480::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:58:39,463::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:40,464::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:40,893::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:40,894::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-597::DEBUG::2014-10-15 06:58:40,895::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:40,896::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:40,896::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-598::DEBUG::2014-10-15 06:58:40,897::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-598::DEBUG::2014-10-15 06:58:40,898::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:41,465::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:42,466::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:43,468::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:43,484::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:43,484::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:43,484::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:43,949::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:43,949::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-599::DEBUG::2014-10-15 06:58:43,950::task::595::Storage.TaskManager.Task::(_updateState) Task=`393aff10-984e-4c0e-b9f6-6aabea68811d`::moving from state init -> state preparing >Thread-599::INFO::2014-10-15 06:58:43,950::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-599::INFO::2014-10-15 06:58:43,951::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000402072', 'lastCheck': '7.2', 'valid': True}} >Thread-599::DEBUG::2014-10-15 06:58:43,951::task::1191::Storage.TaskManager.Task::(prepare) Task=`393aff10-984e-4c0e-b9f6-6aabea68811d`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000402072', 'lastCheck': '7.2', 'valid': True}} >Thread-599::DEBUG::2014-10-15 06:58:43,951::task::595::Storage.TaskManager.Task::(_updateState) Task=`393aff10-984e-4c0e-b9f6-6aabea68811d`::moving from state preparing -> state finished >Thread-599::DEBUG::2014-10-15 06:58:43,951::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-599::DEBUG::2014-10-15 06:58:43,951::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-599::DEBUG::2014-10-15 06:58:43,951::task::993::Storage.TaskManager.Task::(_decref) Task=`393aff10-984e-4c0e-b9f6-6aabea68811d`::ref 0 aborting False >Thread-599::ERROR::2014-10-15 06:58:43,952::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-599::DEBUG::2014-10-15 06:58:43,953::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:43,963::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:43,964::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-600::DEBUG::2014-10-15 06:58:43,965::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-600::DEBUG::2014-10-15 06:58:43,965::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:44,469::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:45,470::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:46,471::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:58:46,727::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:46,728::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:46,728::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:46,728::__init__::232::IOProcess::(_processLogs) (491) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:58:46,728::__init__::232::IOProcess::(_processLogs) (491) Queuing response >Thread-25::DEBUG::2014-10-15 06:58:46,728::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:58:46,735::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00038722 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:58:46,735::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:46,736::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:46,736::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:46,736::__init__::232::IOProcess::(_processLogs) (492) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:58:46,736::__init__::232::IOProcess::(_processLogs) (492) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:46,737::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:46,737::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:46,737::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:46,737::__init__::232::IOProcess::(_processLogs) (493) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:46,737::__init__::232::IOProcess::(_processLogs) (493) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:46,737::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:46,737::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:46,738::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:46,738::__init__::232::IOProcess::(_processLogs) (494) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:46,738::__init__::232::IOProcess::(_processLogs) (494) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:46,738::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:46,738::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:46,738::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:46,738::__init__::232::IOProcess::(_processLogs) (495) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:46,738::__init__::232::IOProcess::(_processLogs) (495) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:47,117::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:47,118::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-601::DEBUG::2014-10-15 06:58:47,119::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:47,120::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:47,120::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-602::DEBUG::2014-10-15 06:58:47,121::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-602::DEBUG::2014-10-15 06:58:47,122::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:47,473::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:48,474::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:48,490::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:48,491::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:48,491::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:58:49,475::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:50,156::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:50,157::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-603::DEBUG::2014-10-15 06:58:50,158::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:50,159::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:50,160::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-604::DEBUG::2014-10-15 06:58:50,160::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-604::DEBUG::2014-10-15 06:58:50,161::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:50,476::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:51,478::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:52,479::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:53,181::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:53,182::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-605::DEBUG::2014-10-15 06:58:53,183::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:53,184::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:53,184::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-606::DEBUG::2014-10-15 06:58:53,185::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-606::DEBUG::2014-10-15 06:58:53,186::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:53,480::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:53,497::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:53,497::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:53,498::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:58:54,481::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:55,483::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:56,210::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:56,210::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-607::DEBUG::2014-10-15 06:58:56,211::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:56,212::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:56,213::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-608::DEBUG::2014-10-15 06:58:56,214::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-608::DEBUG::2014-10-15 06:58:56,214::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:56,484::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:58:56,739::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:56,740::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:56,740::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:56,740::__init__::232::IOProcess::(_processLogs) (496) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:58:56,740::__init__::232::IOProcess::(_processLogs) (496) Queuing response >Thread-25::DEBUG::2014-10-15 06:58:56,740::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:58:56,746::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000406602 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:58:56,747::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:56,747::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:56,747::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:56,748::__init__::232::IOProcess::(_processLogs) (497) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:58:56,748::__init__::232::IOProcess::(_processLogs) (497) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:56,748::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:56,748::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:56,748::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:56,749::__init__::232::IOProcess::(_processLogs) (498) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:56,749::__init__::232::IOProcess::(_processLogs) (498) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:56,749::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:56,749::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:56,749::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:56,749::__init__::232::IOProcess::(_processLogs) (499) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:56,749::__init__::232::IOProcess::(_processLogs) (499) Queuing response >Thread-16::DEBUG::2014-10-15 06:58:56,750::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:58:56,750::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:58:56,750::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:58:56,750::__init__::232::IOProcess::(_processLogs) (500) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:58:56,750::__init__::232::IOProcess::(_processLogs) (500) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:58:57,485::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:58:58,486::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:58,503::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:58,504::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:58:58,504::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:59,230::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:59,231::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-609::DEBUG::2014-10-15 06:58:59,232::task::595::Storage.TaskManager.Task::(_updateState) Task=`f219c933-dbd1-41d1-a881-3966b52bc30b`::moving from state init -> state preparing >Thread-609::INFO::2014-10-15 06:58:59,233::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-609::INFO::2014-10-15 06:58:59,233::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000406602', 'lastCheck': '2.5', 'valid': True}} >Thread-609::DEBUG::2014-10-15 06:58:59,233::task::1191::Storage.TaskManager.Task::(prepare) Task=`f219c933-dbd1-41d1-a881-3966b52bc30b`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000406602', 'lastCheck': '2.5', 'valid': True}} >Thread-609::DEBUG::2014-10-15 06:58:59,233::task::595::Storage.TaskManager.Task::(_updateState) Task=`f219c933-dbd1-41d1-a881-3966b52bc30b`::moving from state preparing -> state finished >Thread-609::DEBUG::2014-10-15 06:58:59,233::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-609::DEBUG::2014-10-15 06:58:59,233::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-609::DEBUG::2014-10-15 06:58:59,233::task::993::Storage.TaskManager.Task::(_decref) Task=`f219c933-dbd1-41d1-a881-3966b52bc30b`::ref 0 aborting False >Thread-609::ERROR::2014-10-15 06:58:59,234::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-609::DEBUG::2014-10-15 06:58:59,235::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:58:59,244::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:58:59,244::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-610::DEBUG::2014-10-15 06:58:59,245::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-610::DEBUG::2014-10-15 06:58:59,245::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:58:59,488::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:00,489::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:01,490::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:02,341::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:02,343::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-611::DEBUG::2014-10-15 06:59:02,345::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:02,348::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:02,349::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-612::DEBUG::2014-10-15 06:59:02,351::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-612::DEBUG::2014-10-15 06:59:02,352::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:02,492::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:03,493::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:03,510::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:03,511::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:03,511::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 06:59:03,601::task::595::Storage.TaskManager.Task::(_updateState) Task=`da015e85-5122-44cd-a842-45d8d57a4920`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 06:59:03,601::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 06:59:03,602::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:03,602::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:03,602::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:03,602::__init__::232::IOProcess::(_processLogs) (501) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:59:03,603::__init__::232::IOProcess::(_processLogs) (501) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:03,604::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:03,604::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:03,604::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:03,604::__init__::232::IOProcess::(_processLogs) (502) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 06:59:03,604::__init__::232::IOProcess::(_processLogs) (502) Queuing response >Thread-17::INFO::2014-10-15 06:59:03,604::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:59:03,605::task::1191::Storage.TaskManager.Task::(prepare) Task=`da015e85-5122-44cd-a842-45d8d57a4920`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 06:59:03,605::task::595::Storage.TaskManager.Task::(_updateState) Task=`da015e85-5122-44cd-a842-45d8d57a4920`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 06:59:03,605::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 06:59:03,605::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 06:59:03,605::task::993::Storage.TaskManager.Task::(_decref) Task=`da015e85-5122-44cd-a842-45d8d57a4920`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 06:59:04,495::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:05,404::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:05,404::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-613::DEBUG::2014-10-15 06:59:05,405::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:05,406::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:05,407::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-614::DEBUG::2014-10-15 06:59:05,407::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-614::DEBUG::2014-10-15 06:59:05,408::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:05,496::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:06,497::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:59:06,751::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:06,751::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:06,751::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:06,752::__init__::232::IOProcess::(_processLogs) (503) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:59:06,752::__init__::232::IOProcess::(_processLogs) (503) Queuing response >Thread-25::DEBUG::2014-10-15 06:59:06,752::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:59:06,759::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000435064 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:59:06,760::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:06,760::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:06,760::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:06,760::__init__::232::IOProcess::(_processLogs) (504) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:59:06,760::__init__::232::IOProcess::(_processLogs) (504) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:06,761::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:06,761::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:06,761::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:06,761::__init__::232::IOProcess::(_processLogs) (505) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:06,761::__init__::232::IOProcess::(_processLogs) (505) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:06,762::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:06,762::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:06,762::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:06,762::__init__::232::IOProcess::(_processLogs) (506) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:06,762::__init__::232::IOProcess::(_processLogs) (506) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:06,762::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:06,762::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:06,763::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:06,763::__init__::232::IOProcess::(_processLogs) (507) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:06,763::__init__::232::IOProcess::(_processLogs) (507) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:59:07,498::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:08,428::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:08,429::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-615::DEBUG::2014-10-15 06:59:08,430::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:08,431::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:08,432::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-616::DEBUG::2014-10-15 06:59:08,432::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-616::DEBUG::2014-10-15 06:59:08,433::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:08,500::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:08,500::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:59:08,500::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:08,517::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:08,517::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:08,518::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:59:09,501::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:10,503::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:11,449::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:11,450::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-617::DEBUG::2014-10-15 06:59:11,451::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:11,452::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:11,452::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-618::DEBUG::2014-10-15 06:59:11,453::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-618::DEBUG::2014-10-15 06:59:11,454::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:11,504::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:12,505::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:13,506::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:13,524::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:13,524::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:13,524::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:14,472::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:14,473::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-619::DEBUG::2014-10-15 06:59:14,474::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed17df26-4ab5-4c0b-a95d-998ef5330b76`::moving from state init -> state preparing >Thread-619::INFO::2014-10-15 06:59:14,474::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-619::INFO::2014-10-15 06:59:14,474::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000435064', 'lastCheck': '7.7', 'valid': True}} >Thread-619::DEBUG::2014-10-15 06:59:14,474::task::1191::Storage.TaskManager.Task::(prepare) Task=`ed17df26-4ab5-4c0b-a95d-998ef5330b76`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000435064', 'lastCheck': '7.7', 'valid': True}} >Thread-619::DEBUG::2014-10-15 06:59:14,474::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed17df26-4ab5-4c0b-a95d-998ef5330b76`::moving from state preparing -> state finished >Thread-619::DEBUG::2014-10-15 06:59:14,475::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-619::DEBUG::2014-10-15 06:59:14,475::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-619::DEBUG::2014-10-15 06:59:14,475::task::993::Storage.TaskManager.Task::(_decref) Task=`ed17df26-4ab5-4c0b-a95d-998ef5330b76`::ref 0 aborting False >Thread-619::ERROR::2014-10-15 06:59:14,476::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-619::DEBUG::2014-10-15 06:59:14,476::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:14,483::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:14,484::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-620::DEBUG::2014-10-15 06:59:14,485::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-620::DEBUG::2014-10-15 06:59:14,485::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:14,508::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:15,509::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:16,510::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:59:16,764::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:16,764::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:16,764::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:16,764::__init__::232::IOProcess::(_processLogs) (508) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:59:16,764::__init__::232::IOProcess::(_processLogs) (508) Queuing response >Thread-25::DEBUG::2014-10-15 06:59:16,765::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:59:16,771::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00038879 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:59:16,771::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:16,771::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:16,771::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:16,772::__init__::232::IOProcess::(_processLogs) (509) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:59:16,772::__init__::232::IOProcess::(_processLogs) (509) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:16,772::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:16,772::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:16,772::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:16,772::__init__::232::IOProcess::(_processLogs) (510) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:16,773::__init__::232::IOProcess::(_processLogs) (510) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:16,773::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:16,773::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:16,773::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:16,773::__init__::232::IOProcess::(_processLogs) (511) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:16,773::__init__::232::IOProcess::(_processLogs) (511) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:16,774::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:16,774::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:16,774::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:16,774::__init__::232::IOProcess::(_processLogs) (512) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:16,774::__init__::232::IOProcess::(_processLogs) (512) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:59:17,511::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:17,607::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:17,608::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-621::DEBUG::2014-10-15 06:59:17,609::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:17,610::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:17,610::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-622::DEBUG::2014-10-15 06:59:17,611::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-622::DEBUG::2014-10-15 06:59:17,612::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:18,513::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:18,530::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:18,531::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:18,531::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:59:19,514::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:20,515::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:20,642::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:20,643::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-623::DEBUG::2014-10-15 06:59:20,644::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:20,645::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:20,645::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-624::DEBUG::2014-10-15 06:59:20,646::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-624::DEBUG::2014-10-15 06:59:20,647::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:21,516::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:22,518::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:23,519::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:23,537::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:23,537::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:23,538::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:23,780::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:23,781::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-625::DEBUG::2014-10-15 06:59:23,782::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:23,783::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:23,783::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-626::DEBUG::2014-10-15 06:59:23,784::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-626::DEBUG::2014-10-15 06:59:23,785::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:24,520::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:25,521::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:26,523::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-25::DEBUG::2014-10-15 06:59:26,775::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 06:59:26,775::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:26,776::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:26,776::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:26,776::__init__::232::IOProcess::(_processLogs) (513) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:59:26,776::__init__::232::IOProcess::(_processLogs) (513) Queuing response >Thread-25::DEBUG::2014-10-15 06:59:26,783::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 06:59:26,784::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:26,784::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:26,784::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:26,784::__init__::232::IOProcess::(_processLogs) (514) Got request for method 'touch' >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:26,811::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:26,812::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-628::DEBUG::2014-10-15 06:59:26,813::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 06:59:26,813::__init__::232::IOProcess::(_processLogs) (514) Queuing response >Thread-25::DEBUG::2014-10-15 06:59:26,813::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:26,814::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-16::DEBUG::2014-10-15 06:59:26,814::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:26,815::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >JsonRpcServer::DEBUG::2014-10-15 06:59:26,816::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-16::DEBUG::2014-10-15 06:59:26,816::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:26,817::__init__::232::IOProcess::(_processLogs) (515) Got request for method 'readfile' >Thread-629::DEBUG::2014-10-15 06:59:26,818::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-16::DEBUG::2014-10-15 06:59:26,818::__init__::232::IOProcess::(_processLogs) (515) Queuing response >Thread-25::DEBUG::2014-10-15 06:59:26,819::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=domain1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-0000000003e1', 'REMOTE_PATH=10.66.128.104:/home/NFS1', 'ROLE=Master', 'SDUUID=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b90f33781d11f6e6b0b3945de3167c0719a458c1'] >Thread-629::DEBUG::2014-10-15 06:59:26,819::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 06:59:26,821::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:26,821::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:26,821::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:26,821::__init__::232::IOProcess::(_processLogs) (516) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:26,821::__init__::232::IOProcess::(_processLogs) (516) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:26,822::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:26,822::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:26,822::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:26,822::__init__::232::IOProcess::(_processLogs) (517) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 06:59:26,822::__init__::232::IOProcess::(_processLogs) (517) Queuing response >Thread-25::DEBUG::2014-10-15 06:59:26,822::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-25::INFO::2014-10-15 06:59:26,822::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_imageNS already registered >Thread-25::INFO::2014-10-15 06:59:26,822::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_volumeNS already registered >Thread-16::DEBUG::2014-10-15 06:59:26,823::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:26,823::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:26,823::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:26,823::__init__::232::IOProcess::(_processLogs) (518) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:59:26,823::__init__::232::IOProcess::(_processLogs) (518) Queuing response >Thread-25::DEBUG::2014-10-15 06:59:26,823::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:59:26,829::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000342682 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:59:26,830::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:26,830::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:26,830::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:26,830::__init__::232::IOProcess::(_processLogs) (519) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:59:26,831::__init__::232::IOProcess::(_processLogs) (519) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:26,831::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:26,832::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:26,832::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:26,832::__init__::232::IOProcess::(_processLogs) (520) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:26,833::__init__::232::IOProcess::(_processLogs) (520) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:26,833::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:26,834::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:26,834::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:26,834::__init__::232::IOProcess::(_processLogs) (521) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:26,836::__init__::232::IOProcess::(_processLogs) (521) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:26,836::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:26,836::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:26,836::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:26,836::__init__::232::IOProcess::(_processLogs) (522) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:26,839::__init__::232::IOProcess::(_processLogs) (522) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:59:27,524::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:28,526::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:28,542::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:28,542::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:28,542::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:59:29,527::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:29,839::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:29,840::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-630::DEBUG::2014-10-15 06:59:29,841::task::595::Storage.TaskManager.Task::(_updateState) Task=`9a8b9850-2e4f-4f92-b7bf-bab21e9750ab`::moving from state init -> state preparing >Thread-630::INFO::2014-10-15 06:59:29,841::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-630::INFO::2014-10-15 06:59:29,841::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000342682', 'lastCheck': '3.0', 'valid': True}} >Thread-630::DEBUG::2014-10-15 06:59:29,841::task::1191::Storage.TaskManager.Task::(prepare) Task=`9a8b9850-2e4f-4f92-b7bf-bab21e9750ab`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000342682', 'lastCheck': '3.0', 'valid': True}} >Thread-630::DEBUG::2014-10-15 06:59:29,841::task::595::Storage.TaskManager.Task::(_updateState) Task=`9a8b9850-2e4f-4f92-b7bf-bab21e9750ab`::moving from state preparing -> state finished >Thread-630::DEBUG::2014-10-15 06:59:29,842::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-630::DEBUG::2014-10-15 06:59:29,842::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-630::DEBUG::2014-10-15 06:59:29,842::task::993::Storage.TaskManager.Task::(_decref) Task=`9a8b9850-2e4f-4f92-b7bf-bab21e9750ab`::ref 0 aborting False >Thread-630::ERROR::2014-10-15 06:59:29,843::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-630::DEBUG::2014-10-15 06:59:29,843::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:29,854::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:29,854::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-631::DEBUG::2014-10-15 06:59:29,855::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-631::DEBUG::2014-10-15 06:59:29,856::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:30,528::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:31,530::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:32,531::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:32,973::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:32,974::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-632::DEBUG::2014-10-15 06:59:32,975::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:32,976::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:32,976::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-633::DEBUG::2014-10-15 06:59:32,977::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-633::DEBUG::2014-10-15 06:59:32,978::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:33,533::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:33,548::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:33,549::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:33,550::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:59:34,534::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:35,535::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:36,028::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:36,029::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-634::DEBUG::2014-10-15 06:59:36,030::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:36,031::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:36,031::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-635::DEBUG::2014-10-15 06:59:36,032::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-635::DEBUG::2014-10-15 06:59:36,032::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:36,536::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:59:36,840::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:36,840::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:36,840::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:36,840::__init__::232::IOProcess::(_processLogs) (523) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:59:36,840::__init__::232::IOProcess::(_processLogs) (523) Queuing response >Thread-25::DEBUG::2014-10-15 06:59:36,841::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:59:36,847::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000363506 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:59:36,847::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:36,847::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:36,847::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:36,848::__init__::232::IOProcess::(_processLogs) (524) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:59:36,848::__init__::232::IOProcess::(_processLogs) (524) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:36,848::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:36,848::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:36,848::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:36,849::__init__::232::IOProcess::(_processLogs) (525) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:36,849::__init__::232::IOProcess::(_processLogs) (525) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:36,849::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:36,849::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:36,849::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:36,849::__init__::232::IOProcess::(_processLogs) (526) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:36,850::__init__::232::IOProcess::(_processLogs) (526) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:36,850::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:36,850::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:36,850::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:36,850::__init__::232::IOProcess::(_processLogs) (527) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:36,850::__init__::232::IOProcess::(_processLogs) (527) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:59:37,537::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:38,538::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:38,538::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 06:59:38,538::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:38,555::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:38,556::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:38,556::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:39,054::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:39,054::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-636::DEBUG::2014-10-15 06:59:39,055::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:39,056::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:39,057::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-637::DEBUG::2014-10-15 06:59:39,058::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-637::DEBUG::2014-10-15 06:59:39,058::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:39,540::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:40,541::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:41,542::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:42,077::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:42,078::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-638::DEBUG::2014-10-15 06:59:42,079::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:42,080::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:42,081::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-639::DEBUG::2014-10-15 06:59:42,082::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-639::DEBUG::2014-10-15 06:59:42,082::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:42,543::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:43,545::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:43,562::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:43,563::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:43,563::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:59:44,546::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:45,098::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:45,099::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-640::DEBUG::2014-10-15 06:59:45,100::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7a0f562-8831-4788-bd6e-72d94d7b3edc`::moving from state init -> state preparing >Thread-640::INFO::2014-10-15 06:59:45,100::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-640::INFO::2014-10-15 06:59:45,100::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000363506', 'lastCheck': '8.2', 'valid': True}} >Thread-640::DEBUG::2014-10-15 06:59:45,100::task::1191::Storage.TaskManager.Task::(prepare) Task=`b7a0f562-8831-4788-bd6e-72d94d7b3edc`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000363506', 'lastCheck': '8.2', 'valid': True}} >Thread-640::DEBUG::2014-10-15 06:59:45,101::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7a0f562-8831-4788-bd6e-72d94d7b3edc`::moving from state preparing -> state finished >Thread-640::DEBUG::2014-10-15 06:59:45,101::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-640::DEBUG::2014-10-15 06:59:45,101::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-640::DEBUG::2014-10-15 06:59:45,101::task::993::Storage.TaskManager.Task::(_decref) Task=`b7a0f562-8831-4788-bd6e-72d94d7b3edc`::ref 0 aborting False >Thread-640::ERROR::2014-10-15 06:59:45,102::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-640::DEBUG::2014-10-15 06:59:45,102::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:45,114::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:45,114::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-641::DEBUG::2014-10-15 06:59:45,115::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-641::DEBUG::2014-10-15 06:59:45,116::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:45,547::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:46,548::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:59:46,851::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:46,851::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:46,852::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:46,852::__init__::232::IOProcess::(_processLogs) (528) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:59:46,852::__init__::232::IOProcess::(_processLogs) (528) Queuing response >Thread-25::DEBUG::2014-10-15 06:59:46,852::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:59:46,858::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000446197 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:59:46,859::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:46,859::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:46,859::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:46,859::__init__::232::IOProcess::(_processLogs) (529) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:59:46,859::__init__::232::IOProcess::(_processLogs) (529) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:46,860::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:46,860::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:46,860::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:46,860::__init__::232::IOProcess::(_processLogs) (530) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:46,860::__init__::232::IOProcess::(_processLogs) (530) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:46,861::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:46,861::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:46,861::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:46,861::__init__::232::IOProcess::(_processLogs) (531) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:46,861::__init__::232::IOProcess::(_processLogs) (531) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:46,862::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:46,862::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:46,862::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:46,862::__init__::232::IOProcess::(_processLogs) (532) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:46,862::__init__::232::IOProcess::(_processLogs) (532) Queuing response >VM Channels Listener::DEBUG::2014-10-15 06:59:47,549::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:48,256::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:48,257::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-642::DEBUG::2014-10-15 06:59:48,258::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:48,259::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:48,260::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-643::DEBUG::2014-10-15 06:59:48,261::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-643::DEBUG::2014-10-15 06:59:48,261::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:48,550::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:48,569::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:48,569::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:48,570::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:59:49,551::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:50,553::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:51,294::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:51,294::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-644::DEBUG::2014-10-15 06:59:51,295::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:51,297::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:51,297::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-645::DEBUG::2014-10-15 06:59:51,298::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-645::DEBUG::2014-10-15 06:59:51,298::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:51,554::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:52,555::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:53,557::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:53,575::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:53,576::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:53,576::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:54,321::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:54,322::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-646::DEBUG::2014-10-15 06:59:54,323::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:54,324::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:54,324::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-647::DEBUG::2014-10-15 06:59:54,325::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-647::DEBUG::2014-10-15 06:59:54,326::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:54,558::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:55,559::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:56,560::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 06:59:56,863::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:56,863::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:56,863::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:56,863::__init__::232::IOProcess::(_processLogs) (533) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:59:56,864::__init__::232::IOProcess::(_processLogs) (533) Queuing response >Thread-25::DEBUG::2014-10-15 06:59:56,864::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 06:59:56,870::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000457211 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 06:59:56,871::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:56,871::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:56,871::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:56,871::__init__::232::IOProcess::(_processLogs) (534) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 06:59:56,871::__init__::232::IOProcess::(_processLogs) (534) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:56,872::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:56,872::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:56,872::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:56,872::__init__::232::IOProcess::(_processLogs) (535) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:56,872::__init__::232::IOProcess::(_processLogs) (535) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:56,873::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:56,873::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:56,873::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:56,873::__init__::232::IOProcess::(_processLogs) (536) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:56,873::__init__::232::IOProcess::(_processLogs) (536) Queuing response >Thread-16::DEBUG::2014-10-15 06:59:56,874::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 06:59:56,874::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 06:59:56,874::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 06:59:56,874::__init__::232::IOProcess::(_processLogs) (537) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 06:59:56,874::__init__::232::IOProcess::(_processLogs) (537) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:57,341::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:57,342::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-648::DEBUG::2014-10-15 06:59:57,346::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 06:59:57,348::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 06:59:57,349::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-649::DEBUG::2014-10-15 06:59:57,350::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-649::DEBUG::2014-10-15 06:59:57,350::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 06:59:57,562::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 06:59:58,563::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:58,582::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:58,583::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 06:59:58,583::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 06:59:59,564::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:00,366::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:00,366::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-650::DEBUG::2014-10-15 07:00:00,368::task::595::Storage.TaskManager.Task::(_updateState) Task=`1e5a5c25-1cf1-4ba5-81f6-9c6cb5b1a94f`::moving from state init -> state preparing >Thread-650::INFO::2014-10-15 07:00:00,368::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-650::INFO::2014-10-15 07:00:00,368::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000457211', 'lastCheck': '3.5', 'valid': True}} >Thread-650::DEBUG::2014-10-15 07:00:00,368::task::1191::Storage.TaskManager.Task::(prepare) Task=`1e5a5c25-1cf1-4ba5-81f6-9c6cb5b1a94f`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000457211', 'lastCheck': '3.5', 'valid': True}} >Thread-650::DEBUG::2014-10-15 07:00:00,368::task::595::Storage.TaskManager.Task::(_updateState) Task=`1e5a5c25-1cf1-4ba5-81f6-9c6cb5b1a94f`::moving from state preparing -> state finished >Thread-650::DEBUG::2014-10-15 07:00:00,368::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-650::DEBUG::2014-10-15 07:00:00,368::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-650::DEBUG::2014-10-15 07:00:00,369::task::993::Storage.TaskManager.Task::(_decref) Task=`1e5a5c25-1cf1-4ba5-81f6-9c6cb5b1a94f`::ref 0 aborting False >Thread-650::ERROR::2014-10-15 07:00:00,369::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-650::DEBUG::2014-10-15 07:00:00,370::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:00,379::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:00,380::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-651::DEBUG::2014-10-15 07:00:00,381::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-651::DEBUG::2014-10-15 07:00:00,381::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:00,565::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:01,567::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:02,568::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:03,521::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:03,521::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-652::DEBUG::2014-10-15 07:00:03,523::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:03,524::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:03,524::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-653::DEBUG::2014-10-15 07:00:03,525::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-653::DEBUG::2014-10-15 07:00:03,526::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:03,570::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:03,588::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:03,588::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:03,589::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 07:00:03,643::task::595::Storage.TaskManager.Task::(_updateState) Task=`48f9bc4b-6adb-4b49-a604-82ab33934064`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 07:00:03,643::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:00:03,644::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:03,645::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:03,645::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:03,645::__init__::232::IOProcess::(_processLogs) (538) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:00:03,646::__init__::232::IOProcess::(_processLogs) (538) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:03,646::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:03,646::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:03,647::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:03,647::__init__::232::IOProcess::(_processLogs) (539) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:00:03,647::__init__::232::IOProcess::(_processLogs) (539) Queuing response >Thread-17::INFO::2014-10-15 07:00:03,647::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:00:03,647::task::1191::Storage.TaskManager.Task::(prepare) Task=`48f9bc4b-6adb-4b49-a604-82ab33934064`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:00:03,647::task::595::Storage.TaskManager.Task::(_updateState) Task=`48f9bc4b-6adb-4b49-a604-82ab33934064`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 07:00:03,647::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 07:00:03,647::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 07:00:03,647::task::993::Storage.TaskManager.Task::(_decref) Task=`48f9bc4b-6adb-4b49-a604-82ab33934064`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 07:00:04,571::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:05,572::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:06,561::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:06,561::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-654::DEBUG::2014-10-15 07:00:06,562::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:06,563::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:06,564::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-655::DEBUG::2014-10-15 07:00:06,565::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-655::DEBUG::2014-10-15 07:00:06,565::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:06,573::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:00:06,875::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:06,875::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:06,876::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:06,876::__init__::232::IOProcess::(_processLogs) (540) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:00:06,876::__init__::232::IOProcess::(_processLogs) (540) Queuing response >Thread-25::DEBUG::2014-10-15 07:00:06,876::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:00:06,883::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000421241 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:00:06,884::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:06,884::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:06,884::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:06,884::__init__::232::IOProcess::(_processLogs) (541) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:00:06,884::__init__::232::IOProcess::(_processLogs) (541) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:06,885::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:06,885::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:06,885::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:06,885::__init__::232::IOProcess::(_processLogs) (542) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:06,885::__init__::232::IOProcess::(_processLogs) (542) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:06,886::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:06,886::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:06,886::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:06,886::__init__::232::IOProcess::(_processLogs) (543) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:06,886::__init__::232::IOProcess::(_processLogs) (543) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:06,886::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:06,887::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:06,887::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:06,887::__init__::232::IOProcess::(_processLogs) (544) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:06,887::__init__::232::IOProcess::(_processLogs) (544) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:00:07,575::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:08,576::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:08,576::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:00:08,577::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:08,594::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:08,595::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:08,595::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:00:09,578::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:09,586::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:09,587::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-656::DEBUG::2014-10-15 07:00:09,588::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:09,589::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:09,589::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-657::DEBUG::2014-10-15 07:00:09,590::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-657::DEBUG::2014-10-15 07:00:09,591::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:10,579::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:11,580::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:12,582::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:12,611::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:12,612::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-658::DEBUG::2014-10-15 07:00:12,613::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:12,614::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:12,615::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-659::DEBUG::2014-10-15 07:00:12,616::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-659::DEBUG::2014-10-15 07:00:12,616::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:13,583::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:13,601::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:13,602::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:13,602::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:00:14,584::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:15,585::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:15,632::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:15,633::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-660::DEBUG::2014-10-15 07:00:15,634::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5474643-3392-47f4-9b34-fc75b7607ef8`::moving from state init -> state preparing >Thread-660::INFO::2014-10-15 07:00:15,634::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-660::INFO::2014-10-15 07:00:15,635::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421241', 'lastCheck': '8.7', 'valid': True}} >Thread-660::DEBUG::2014-10-15 07:00:15,635::task::1191::Storage.TaskManager.Task::(prepare) Task=`d5474643-3392-47f4-9b34-fc75b7607ef8`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421241', 'lastCheck': '8.7', 'valid': True}} >Thread-660::DEBUG::2014-10-15 07:00:15,635::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5474643-3392-47f4-9b34-fc75b7607ef8`::moving from state preparing -> state finished >Thread-660::DEBUG::2014-10-15 07:00:15,635::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-660::DEBUG::2014-10-15 07:00:15,635::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-660::DEBUG::2014-10-15 07:00:15,635::task::993::Storage.TaskManager.Task::(_decref) Task=`d5474643-3392-47f4-9b34-fc75b7607ef8`::ref 0 aborting False >Thread-660::ERROR::2014-10-15 07:00:15,636::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-660::DEBUG::2014-10-15 07:00:15,637::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:15,644::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:15,644::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-661::DEBUG::2014-10-15 07:00:15,645::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-661::DEBUG::2014-10-15 07:00:15,646::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:16,587::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:00:16,888::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:16,888::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:16,888::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:16,888::__init__::232::IOProcess::(_processLogs) (545) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:00:16,888::__init__::232::IOProcess::(_processLogs) (545) Queuing response >Thread-25::DEBUG::2014-10-15 07:00:16,889::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:00:16,895::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000409826 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:00:16,896::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:16,896::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:16,896::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:16,896::__init__::232::IOProcess::(_processLogs) (546) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:00:16,896::__init__::232::IOProcess::(_processLogs) (546) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:16,897::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:16,897::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:16,897::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:16,897::__init__::232::IOProcess::(_processLogs) (547) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:16,897::__init__::232::IOProcess::(_processLogs) (547) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:16,898::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:16,898::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:16,898::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:16,898::__init__::232::IOProcess::(_processLogs) (548) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:16,898::__init__::232::IOProcess::(_processLogs) (548) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:16,898::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:16,898::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:16,898::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:16,899::__init__::232::IOProcess::(_processLogs) (549) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:16,899::__init__::232::IOProcess::(_processLogs) (549) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:00:17,588::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:18,589::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:18,608::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:18,608::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:18,609::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:18,815::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:18,816::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-662::DEBUG::2014-10-15 07:00:18,817::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:18,818::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:18,819::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-663::DEBUG::2014-10-15 07:00:18,820::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-663::DEBUG::2014-10-15 07:00:18,820::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:19,591::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:20,592::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:21,593::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:21,852::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:21,852::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-664::DEBUG::2014-10-15 07:00:21,853::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:21,854::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:21,855::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-665::DEBUG::2014-10-15 07:00:21,855::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-665::DEBUG::2014-10-15 07:00:21,856::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:22,595::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:23,596::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:23,610::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:23,611::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:23,611::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:00:24,597::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:24,873::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:24,874::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-666::DEBUG::2014-10-15 07:00:24,875::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:24,876::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:24,877::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-667::DEBUG::2014-10-15 07:00:24,877::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-667::DEBUG::2014-10-15 07:00:24,878::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:25,598::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:26,600::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:00:26,900::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:26,900::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:26,900::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:26,900::__init__::232::IOProcess::(_processLogs) (550) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:00:26,900::__init__::232::IOProcess::(_processLogs) (550) Queuing response >Thread-25::DEBUG::2014-10-15 07:00:26,900::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:00:26,907::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000369193 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:00:26,907::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:26,907::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:26,908::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:26,908::__init__::232::IOProcess::(_processLogs) (551) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:00:26,908::__init__::232::IOProcess::(_processLogs) (551) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:26,908::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:26,908::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:26,909::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:26,909::__init__::232::IOProcess::(_processLogs) (552) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:26,909::__init__::232::IOProcess::(_processLogs) (552) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:26,909::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:26,909::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:26,909::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:26,909::__init__::232::IOProcess::(_processLogs) (553) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:26,909::__init__::232::IOProcess::(_processLogs) (553) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:26,910::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:26,910::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:26,910::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:26,910::__init__::232::IOProcess::(_processLogs) (554) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:26,910::__init__::232::IOProcess::(_processLogs) (554) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:00:27,601::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:27,898::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:27,899::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-668::DEBUG::2014-10-15 07:00:27,900::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:27,901::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:27,902::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-669::DEBUG::2014-10-15 07:00:27,903::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-669::DEBUG::2014-10-15 07:00:27,903::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:28,602::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:28,617::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:28,617::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:28,618::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:00:29,604::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:30,605::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:30,923::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:30,924::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-670::DEBUG::2014-10-15 07:00:30,925::task::595::Storage.TaskManager.Task::(_updateState) Task=`f8b4fb2c-3797-4119-9ff1-8dde5d5c1f5c`::moving from state init -> state preparing >Thread-670::INFO::2014-10-15 07:00:30,925::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-670::INFO::2014-10-15 07:00:30,925::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000369193', 'lastCheck': '4.0', 'valid': True}} >Thread-670::DEBUG::2014-10-15 07:00:30,925::task::1191::Storage.TaskManager.Task::(prepare) Task=`f8b4fb2c-3797-4119-9ff1-8dde5d5c1f5c`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000369193', 'lastCheck': '4.0', 'valid': True}} >Thread-670::DEBUG::2014-10-15 07:00:30,926::task::595::Storage.TaskManager.Task::(_updateState) Task=`f8b4fb2c-3797-4119-9ff1-8dde5d5c1f5c`::moving from state preparing -> state finished >Thread-670::DEBUG::2014-10-15 07:00:30,926::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-670::DEBUG::2014-10-15 07:00:30,926::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-670::DEBUG::2014-10-15 07:00:30,926::task::993::Storage.TaskManager.Task::(_decref) Task=`f8b4fb2c-3797-4119-9ff1-8dde5d5c1f5c`::ref 0 aborting False >Thread-670::ERROR::2014-10-15 07:00:30,927::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-670::DEBUG::2014-10-15 07:00:30,927::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:30,938::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:30,939::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-671::DEBUG::2014-10-15 07:00:30,939::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-671::DEBUG::2014-10-15 07:00:30,940::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:31,606::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:32,608::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:33,609::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:33,624::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:33,624::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:33,625::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:34,074::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:34,075::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-672::DEBUG::2014-10-15 07:00:34,076::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:34,077::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:34,077::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-673::DEBUG::2014-10-15 07:00:34,078::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-673::DEBUG::2014-10-15 07:00:34,079::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:34,611::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:35,612::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:36,613::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:00:36,911::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:36,911::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:36,911::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:36,912::__init__::232::IOProcess::(_processLogs) (555) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:00:36,912::__init__::232::IOProcess::(_processLogs) (555) Queuing response >Thread-25::DEBUG::2014-10-15 07:00:36,912::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:00:36,918::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000472914 s, 971 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:00:36,919::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:36,919::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:36,919::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:36,919::__init__::232::IOProcess::(_processLogs) (556) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:00:36,919::__init__::232::IOProcess::(_processLogs) (556) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:36,920::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:36,920::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:36,920::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:36,920::__init__::232::IOProcess::(_processLogs) (557) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:36,920::__init__::232::IOProcess::(_processLogs) (557) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:36,921::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:36,921::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:36,921::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:36,921::__init__::232::IOProcess::(_processLogs) (558) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:36,921::__init__::232::IOProcess::(_processLogs) (558) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:36,921::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:36,922::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:36,922::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:36,922::__init__::232::IOProcess::(_processLogs) (559) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:36,922::__init__::232::IOProcess::(_processLogs) (559) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:37,135::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:37,136::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-674::DEBUG::2014-10-15 07:00:37,137::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:37,138::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:37,139::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-675::DEBUG::2014-10-15 07:00:37,139::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-675::DEBUG::2014-10-15 07:00:37,140::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:37,614::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:38,616::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:38,616::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:00:38,616::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:38,630::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:38,631::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:38,631::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:00:39,617::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:40,162::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:40,163::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-676::DEBUG::2014-10-15 07:00:40,164::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:40,165::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:40,165::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-677::DEBUG::2014-10-15 07:00:40,166::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-677::DEBUG::2014-10-15 07:00:40,166::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:40,619::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:41,620::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:42,621::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:43,188::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:43,189::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-678::DEBUG::2014-10-15 07:00:43,190::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:43,191::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:43,191::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-679::DEBUG::2014-10-15 07:00:43,192::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-679::DEBUG::2014-10-15 07:00:43,193::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:43,622::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:43,637::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:43,638::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:43,638::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:00:44,624::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:45,625::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:46,207::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:46,208::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-680::DEBUG::2014-10-15 07:00:46,209::task::595::Storage.TaskManager.Task::(_updateState) Task=`fbd537a6-6497-436e-8bec-381d706e026f`::moving from state init -> state preparing >Thread-680::INFO::2014-10-15 07:00:46,209::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-680::INFO::2014-10-15 07:00:46,210::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000472914', 'lastCheck': '9.3', 'valid': True}} >Thread-680::DEBUG::2014-10-15 07:00:46,210::task::1191::Storage.TaskManager.Task::(prepare) Task=`fbd537a6-6497-436e-8bec-381d706e026f`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000472914', 'lastCheck': '9.3', 'valid': True}} >Thread-680::DEBUG::2014-10-15 07:00:46,210::task::595::Storage.TaskManager.Task::(_updateState) Task=`fbd537a6-6497-436e-8bec-381d706e026f`::moving from state preparing -> state finished >Thread-680::DEBUG::2014-10-15 07:00:46,210::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-680::DEBUG::2014-10-15 07:00:46,210::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-680::DEBUG::2014-10-15 07:00:46,210::task::993::Storage.TaskManager.Task::(_decref) Task=`fbd537a6-6497-436e-8bec-381d706e026f`::ref 0 aborting False >Thread-680::ERROR::2014-10-15 07:00:46,211::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-680::DEBUG::2014-10-15 07:00:46,212::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:46,224::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:46,224::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-681::DEBUG::2014-10-15 07:00:46,225::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-681::DEBUG::2014-10-15 07:00:46,226::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:46,626::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:00:46,923::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:46,923::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:46,923::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:46,923::__init__::232::IOProcess::(_processLogs) (560) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:00:46,923::__init__::232::IOProcess::(_processLogs) (560) Queuing response >Thread-25::DEBUG::2014-10-15 07:00:46,924::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:00:46,930::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000367455 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:00:46,930::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:46,930::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:46,931::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:46,931::__init__::232::IOProcess::(_processLogs) (561) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:00:46,931::__init__::232::IOProcess::(_processLogs) (561) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:46,931::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:46,932::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:46,932::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:46,932::__init__::232::IOProcess::(_processLogs) (562) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:46,932::__init__::232::IOProcess::(_processLogs) (562) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:46,932::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:46,932::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:46,932::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:46,932::__init__::232::IOProcess::(_processLogs) (563) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:46,933::__init__::232::IOProcess::(_processLogs) (563) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:46,933::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:46,933::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:46,933::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:46,933::__init__::232::IOProcess::(_processLogs) (564) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:46,933::__init__::232::IOProcess::(_processLogs) (564) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:00:47,627::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:48,629::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:48,642::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:48,642::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:48,643::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:49,370::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:49,371::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-682::DEBUG::2014-10-15 07:00:49,372::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:49,373::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:49,374::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-683::DEBUG::2014-10-15 07:00:49,375::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-683::DEBUG::2014-10-15 07:00:49,375::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:49,630::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:50,631::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:51,632::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:52,405::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:52,406::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-684::DEBUG::2014-10-15 07:00:52,407::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:52,408::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:52,409::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-685::DEBUG::2014-10-15 07:00:52,409::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-685::DEBUG::2014-10-15 07:00:52,410::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:52,632::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:53,634::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:53,644::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:53,645::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:53,645::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:00:54,635::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:55,433::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:55,434::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-686::DEBUG::2014-10-15 07:00:55,435::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:55,436::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:55,437::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-687::DEBUG::2014-10-15 07:00:55,438::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-687::DEBUG::2014-10-15 07:00:55,438::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:55,636::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:00:56,637::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:00:56,934::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:56,934::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:56,934::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:56,935::__init__::232::IOProcess::(_processLogs) (565) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:00:56,935::__init__::232::IOProcess::(_processLogs) (565) Queuing response >Thread-25::DEBUG::2014-10-15 07:00:56,935::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:00:56,941::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000423674 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:00:56,941::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:56,942::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:56,942::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:56,942::__init__::232::IOProcess::(_processLogs) (566) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:00:56,942::__init__::232::IOProcess::(_processLogs) (566) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:56,943::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:56,943::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:56,943::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:56,943::__init__::232::IOProcess::(_processLogs) (567) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:56,943::__init__::232::IOProcess::(_processLogs) (567) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:56,943::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:56,943::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:56,943::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:56,944::__init__::232::IOProcess::(_processLogs) (568) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:56,944::__init__::232::IOProcess::(_processLogs) (568) Queuing response >Thread-16::DEBUG::2014-10-15 07:00:56,944::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:00:56,944::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:00:56,944::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:00:56,944::__init__::232::IOProcess::(_processLogs) (569) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:00:56,944::__init__::232::IOProcess::(_processLogs) (569) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:00:57,639::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:58,465::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:58,466::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-688::DEBUG::2014-10-15 07:00:58,467::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:00:58,469::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:00:58,469::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-689::DEBUG::2014-10-15 07:00:58,470::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-689::DEBUG::2014-10-15 07:00:58,470::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:00:58,640::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:58,651::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:58,651::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:00:58,652::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:00:59,640::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:00,641::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:01,497::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:01,498::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-690::DEBUG::2014-10-15 07:01:01,502::task::595::Storage.TaskManager.Task::(_updateState) Task=`f00c8d03-4911-4cc6-b0b8-6707b1ec70ed`::moving from state init -> state preparing >Thread-690::INFO::2014-10-15 07:01:01,502::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-690::INFO::2014-10-15 07:01:01,503::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423674', 'lastCheck': '4.6', 'valid': True}} >Thread-690::DEBUG::2014-10-15 07:01:01,503::task::1191::Storage.TaskManager.Task::(prepare) Task=`f00c8d03-4911-4cc6-b0b8-6707b1ec70ed`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423674', 'lastCheck': '4.6', 'valid': True}} >Thread-690::DEBUG::2014-10-15 07:01:01,504::task::595::Storage.TaskManager.Task::(_updateState) Task=`f00c8d03-4911-4cc6-b0b8-6707b1ec70ed`::moving from state preparing -> state finished >Thread-690::DEBUG::2014-10-15 07:01:01,504::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-690::DEBUG::2014-10-15 07:01:01,505::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-690::DEBUG::2014-10-15 07:01:01,505::task::993::Storage.TaskManager.Task::(_decref) Task=`f00c8d03-4911-4cc6-b0b8-6707b1ec70ed`::ref 0 aborting False >Thread-690::ERROR::2014-10-15 07:01:01,508::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-690::DEBUG::2014-10-15 07:01:01,510::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:01,523::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:01,525::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-691::DEBUG::2014-10-15 07:01:01,527::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-691::DEBUG::2014-10-15 07:01:01,528::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:01,643::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:02,644::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:03,646::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:03,658::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:03,658::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:03,658::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 07:01:03,690::task::595::Storage.TaskManager.Task::(_updateState) Task=`51a896d9-a90b-4b4a-b7af-74cda852faef`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 07:01:03,690::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:01:03,691::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:03,691::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:03,691::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:03,691::__init__::232::IOProcess::(_processLogs) (570) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:01:03,692::__init__::232::IOProcess::(_processLogs) (570) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:03,693::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:03,693::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:03,693::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:03,693::__init__::232::IOProcess::(_processLogs) (571) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:01:03,693::__init__::232::IOProcess::(_processLogs) (571) Queuing response >Thread-17::INFO::2014-10-15 07:01:03,694::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:01:03,694::task::1191::Storage.TaskManager.Task::(prepare) Task=`51a896d9-a90b-4b4a-b7af-74cda852faef`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:01:03,694::task::595::Storage.TaskManager.Task::(_updateState) Task=`51a896d9-a90b-4b4a-b7af-74cda852faef`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 07:01:03,694::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 07:01:03,694::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 07:01:03,694::task::993::Storage.TaskManager.Task::(_decref) Task=`51a896d9-a90b-4b4a-b7af-74cda852faef`::ref 0 aborting False >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:04,645::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:04,646::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >VM Channels Listener::DEBUG::2014-10-15 07:01:04,647::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-692::DEBUG::2014-10-15 07:01:04,647::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:04,648::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:04,649::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-693::DEBUG::2014-10-15 07:01:04,650::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-693::DEBUG::2014-10-15 07:01:04,651::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:05,648::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:06,650::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:01:06,945::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:06,946::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:06,946::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:06,946::__init__::232::IOProcess::(_processLogs) (572) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:01:06,946::__init__::232::IOProcess::(_processLogs) (572) Queuing response >Thread-25::DEBUG::2014-10-15 07:01:06,947::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:01:06,953::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000468933 s, 979 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:01:06,954::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:06,954::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:06,954::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:06,955::__init__::232::IOProcess::(_processLogs) (573) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:01:06,955::__init__::232::IOProcess::(_processLogs) (573) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:06,955::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:06,955::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:06,955::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:06,955::__init__::232::IOProcess::(_processLogs) (574) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:06,956::__init__::232::IOProcess::(_processLogs) (574) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:06,956::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:06,956::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:06,956::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:06,956::__init__::232::IOProcess::(_processLogs) (575) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:06,956::__init__::232::IOProcess::(_processLogs) (575) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:06,957::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:06,957::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:06,957::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:06,957::__init__::232::IOProcess::(_processLogs) (576) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:06,957::__init__::232::IOProcess::(_processLogs) (576) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:01:07,651::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:07,688::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:07,689::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-694::DEBUG::2014-10-15 07:01:07,690::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:07,691::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:07,692::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-695::DEBUG::2014-10-15 07:01:07,693::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-695::DEBUG::2014-10-15 07:01:07,693::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:08,652::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:08,652::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:01:08,653::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:08,664::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:08,665::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:08,665::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:01:09,654::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:10,655::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:10,736::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:10,737::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-696::DEBUG::2014-10-15 07:01:10,738::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:10,739::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:10,739::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-697::DEBUG::2014-10-15 07:01:10,740::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-697::DEBUG::2014-10-15 07:01:10,740::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:11,656::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:12,658::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:13,659::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:13,671::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:13,671::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:13,672::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:13,772::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:13,772::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-698::DEBUG::2014-10-15 07:01:13,773::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:13,774::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:13,775::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-699::DEBUG::2014-10-15 07:01:13,776::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-699::DEBUG::2014-10-15 07:01:13,776::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:14,660::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:15,661::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:16,663::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:16,798::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:16,799::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-700::DEBUG::2014-10-15 07:01:16,800::task::595::Storage.TaskManager.Task::(_updateState) Task=`43a1f2ec-a2f4-45a0-9d9e-827a5ba315fe`::moving from state init -> state preparing >Thread-700::INFO::2014-10-15 07:01:16,800::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-700::INFO::2014-10-15 07:01:16,800::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000468933', 'lastCheck': '9.8', 'valid': True}} >Thread-700::DEBUG::2014-10-15 07:01:16,800::task::1191::Storage.TaskManager.Task::(prepare) Task=`43a1f2ec-a2f4-45a0-9d9e-827a5ba315fe`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000468933', 'lastCheck': '9.8', 'valid': True}} >Thread-700::DEBUG::2014-10-15 07:01:16,801::task::595::Storage.TaskManager.Task::(_updateState) Task=`43a1f2ec-a2f4-45a0-9d9e-827a5ba315fe`::moving from state preparing -> state finished >Thread-700::DEBUG::2014-10-15 07:01:16,801::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-700::DEBUG::2014-10-15 07:01:16,801::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-700::DEBUG::2014-10-15 07:01:16,801::task::993::Storage.TaskManager.Task::(_decref) Task=`43a1f2ec-a2f4-45a0-9d9e-827a5ba315fe`::ref 0 aborting False >Thread-700::ERROR::2014-10-15 07:01:16,802::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-700::DEBUG::2014-10-15 07:01:16,802::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:16,812::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:16,813::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-701::DEBUG::2014-10-15 07:01:16,814::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-701::DEBUG::2014-10-15 07:01:16,814::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:01:16,958::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:16,958::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:16,958::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:16,958::__init__::232::IOProcess::(_processLogs) (577) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:01:16,958::__init__::232::IOProcess::(_processLogs) (577) Queuing response >Thread-25::DEBUG::2014-10-15 07:01:16,959::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:01:16,965::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000385142 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:01:16,965::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:16,966::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:16,966::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:16,966::__init__::232::IOProcess::(_processLogs) (578) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:01:16,966::__init__::232::IOProcess::(_processLogs) (578) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:16,966::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:16,966::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:16,967::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:16,967::__init__::232::IOProcess::(_processLogs) (579) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:16,967::__init__::232::IOProcess::(_processLogs) (579) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:16,967::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:16,967::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:16,967::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:16,967::__init__::232::IOProcess::(_processLogs) (580) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:16,968::__init__::232::IOProcess::(_processLogs) (580) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:16,968::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:16,968::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:16,968::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:16,968::__init__::232::IOProcess::(_processLogs) (581) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:16,968::__init__::232::IOProcess::(_processLogs) (581) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:01:17,664::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:18,665::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:18,676::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:18,676::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:18,677::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:01:19,667::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:19,924::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:19,925::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-702::DEBUG::2014-10-15 07:01:19,926::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:19,927::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:19,927::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-703::DEBUG::2014-10-15 07:01:19,928::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-703::DEBUG::2014-10-15 07:01:19,929::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:20,668::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:21,669::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:22,670::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:22,961::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:22,962::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-704::DEBUG::2014-10-15 07:01:22,963::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:22,964::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:22,965::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-705::DEBUG::2014-10-15 07:01:22,966::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-705::DEBUG::2014-10-15 07:01:22,966::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:23,672::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:23,682::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:23,683::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:23,683::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:01:24,673::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:25,674::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:25,985::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:25,986::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-706::DEBUG::2014-10-15 07:01:25,987::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:25,988::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:25,988::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-707::DEBUG::2014-10-15 07:01:25,989::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-707::DEBUG::2014-10-15 07:01:25,989::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:26,675::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:01:26,969::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:26,969::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:26,970::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:26,970::__init__::232::IOProcess::(_processLogs) (582) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:01:26,970::__init__::232::IOProcess::(_processLogs) (582) Queuing response >Thread-25::DEBUG::2014-10-15 07:01:26,970::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:01:26,976::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000384609 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:01:26,977::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:26,977::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:26,977::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:26,977::__init__::232::IOProcess::(_processLogs) (583) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:01:26,977::__init__::232::IOProcess::(_processLogs) (583) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:26,978::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:26,978::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:26,978::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:26,978::__init__::232::IOProcess::(_processLogs) (584) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:26,978::__init__::232::IOProcess::(_processLogs) (584) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:26,979::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:26,979::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:26,979::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:26,979::__init__::232::IOProcess::(_processLogs) (585) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:26,979::__init__::232::IOProcess::(_processLogs) (585) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:26,979::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:26,979::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:26,980::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:26,980::__init__::232::IOProcess::(_processLogs) (586) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:26,980::__init__::232::IOProcess::(_processLogs) (586) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:01:27,676::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:28,677::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:28,685::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:28,686::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:28,686::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:29,010::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:29,011::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-708::DEBUG::2014-10-15 07:01:29,012::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:29,013::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:29,014::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-709::DEBUG::2014-10-15 07:01:29,014::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-709::DEBUG::2014-10-15 07:01:29,015::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:29,679::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:30,679::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:31,681::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:32,034::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:32,036::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-710::DEBUG::2014-10-15 07:01:32,039::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3cc625d-58ec-4b78-a691-0fcb7927f208`::moving from state init -> state preparing >Thread-710::INFO::2014-10-15 07:01:32,039::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-710::INFO::2014-10-15 07:01:32,040::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000384609', 'lastCheck': '5.1', 'valid': True}} >Thread-710::DEBUG::2014-10-15 07:01:32,040::task::1191::Storage.TaskManager.Task::(prepare) Task=`e3cc625d-58ec-4b78-a691-0fcb7927f208`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000384609', 'lastCheck': '5.1', 'valid': True}} >Thread-710::DEBUG::2014-10-15 07:01:32,041::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3cc625d-58ec-4b78-a691-0fcb7927f208`::moving from state preparing -> state finished >Thread-710::DEBUG::2014-10-15 07:01:32,041::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-710::DEBUG::2014-10-15 07:01:32,042::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-710::DEBUG::2014-10-15 07:01:32,042::task::993::Storage.TaskManager.Task::(_decref) Task=`e3cc625d-58ec-4b78-a691-0fcb7927f208`::ref 0 aborting False >Thread-710::ERROR::2014-10-15 07:01:32,044::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-710::DEBUG::2014-10-15 07:01:32,047::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:32,056::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:32,057::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-711::DEBUG::2014-10-15 07:01:32,060::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-711::DEBUG::2014-10-15 07:01:32,061::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:32,682::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:33,683::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:33,692::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:33,693::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:33,693::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:01:34,685::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:35,180::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:35,181::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-712::DEBUG::2014-10-15 07:01:35,182::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:35,183::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:35,183::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-713::DEBUG::2014-10-15 07:01:35,184::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-713::DEBUG::2014-10-15 07:01:35,185::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:35,686::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:36,687::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:01:36,981::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:36,981::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:36,981::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:36,981::__init__::232::IOProcess::(_processLogs) (587) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:01:36,981::__init__::232::IOProcess::(_processLogs) (587) Queuing response >Thread-25::DEBUG::2014-10-15 07:01:36,982::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:01:36,988::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000390229 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:01:36,989::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:36,989::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:36,989::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:36,989::__init__::232::IOProcess::(_processLogs) (588) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:01:36,989::__init__::232::IOProcess::(_processLogs) (588) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:36,990::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:36,990::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:36,990::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:36,990::__init__::232::IOProcess::(_processLogs) (589) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:36,990::__init__::232::IOProcess::(_processLogs) (589) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:36,990::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:36,991::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:36,991::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:36,991::__init__::232::IOProcess::(_processLogs) (590) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:36,991::__init__::232::IOProcess::(_processLogs) (590) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:36,991::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:36,991::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:36,991::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:36,991::__init__::232::IOProcess::(_processLogs) (591) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:36,992::__init__::232::IOProcess::(_processLogs) (591) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:01:37,688::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:38,273::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:38,274::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-714::DEBUG::2014-10-15 07:01:38,275::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:38,276::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:38,277::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-715::DEBUG::2014-10-15 07:01:38,278::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-715::DEBUG::2014-10-15 07:01:38,278::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:38,690::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:38,690::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:01:38,690::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:38,699::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:38,699::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:38,700::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:01:39,691::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:40,693::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:41,298::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:41,298::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-716::DEBUG::2014-10-15 07:01:41,299::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:41,300::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:41,301::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-717::DEBUG::2014-10-15 07:01:41,302::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-717::DEBUG::2014-10-15 07:01:41,302::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:41,694::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:42,695::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:43,696::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:43,706::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:43,706::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:43,706::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:44,328::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:44,329::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-718::DEBUG::2014-10-15 07:01:44,330::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:44,331::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:44,332::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-719::DEBUG::2014-10-15 07:01:44,333::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-719::DEBUG::2014-10-15 07:01:44,333::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:44,698::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:45,699::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:46,700::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:01:46,992::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:46,993::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:46,993::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:46,993::__init__::232::IOProcess::(_processLogs) (592) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:01:46,993::__init__::232::IOProcess::(_processLogs) (592) Queuing response >Thread-25::DEBUG::2014-10-15 07:01:46,993::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:01:46,999::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000389577 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:01:47,000::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:47,000::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:47,000::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:47,001::__init__::232::IOProcess::(_processLogs) (593) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:01:47,001::__init__::232::IOProcess::(_processLogs) (593) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:47,001::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:47,001::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:47,001::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:47,002::__init__::232::IOProcess::(_processLogs) (594) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:47,002::__init__::232::IOProcess::(_processLogs) (594) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:47,002::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:47,002::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:47,002::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:47,002::__init__::232::IOProcess::(_processLogs) (595) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:47,002::__init__::232::IOProcess::(_processLogs) (595) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:47,003::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:47,003::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:47,003::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:47,003::__init__::232::IOProcess::(_processLogs) (596) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:47,003::__init__::232::IOProcess::(_processLogs) (596) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:47,352::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:47,353::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-720::DEBUG::2014-10-15 07:01:47,354::task::595::Storage.TaskManager.Task::(_updateState) Task=`26028df0-c9fc-4afb-824e-c8543548223e`::moving from state init -> state preparing >Thread-720::INFO::2014-10-15 07:01:47,354::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-720::INFO::2014-10-15 07:01:47,354::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000389577', 'lastCheck': '0.4', 'valid': True}} >Thread-720::DEBUG::2014-10-15 07:01:47,354::task::1191::Storage.TaskManager.Task::(prepare) Task=`26028df0-c9fc-4afb-824e-c8543548223e`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000389577', 'lastCheck': '0.4', 'valid': True}} >Thread-720::DEBUG::2014-10-15 07:01:47,354::task::595::Storage.TaskManager.Task::(_updateState) Task=`26028df0-c9fc-4afb-824e-c8543548223e`::moving from state preparing -> state finished >Thread-720::DEBUG::2014-10-15 07:01:47,354::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-720::DEBUG::2014-10-15 07:01:47,355::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-720::DEBUG::2014-10-15 07:01:47,355::task::993::Storage.TaskManager.Task::(_decref) Task=`26028df0-c9fc-4afb-824e-c8543548223e`::ref 0 aborting False >Thread-720::ERROR::2014-10-15 07:01:47,356::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-720::DEBUG::2014-10-15 07:01:47,356::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:47,366::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:47,366::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-721::DEBUG::2014-10-15 07:01:47,367::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-721::DEBUG::2014-10-15 07:01:47,367::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:47,701::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:48,703::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:48,712::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:48,713::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:48,713::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:01:49,704::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:50,469::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:50,470::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-722::DEBUG::2014-10-15 07:01:50,471::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:50,472::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:50,472::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-723::DEBUG::2014-10-15 07:01:50,473::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-723::DEBUG::2014-10-15 07:01:50,474::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:50,705::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:51,706::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:52,707::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:53,544::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:53,545::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-724::DEBUG::2014-10-15 07:01:53,546::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:53,547::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:53,547::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-725::DEBUG::2014-10-15 07:01:53,548::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-725::DEBUG::2014-10-15 07:01:53,549::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:53,708::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:53,719::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:53,719::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:53,720::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:01:54,709::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:55,710::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:56,568::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:56,569::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-726::DEBUG::2014-10-15 07:01:56,570::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:56,571::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:56,572::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-727::DEBUG::2014-10-15 07:01:56,573::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-727::DEBUG::2014-10-15 07:01:56,573::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:56,712::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:01:57,004::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:57,004::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:57,004::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:57,004::__init__::232::IOProcess::(_processLogs) (597) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:01:57,005::__init__::232::IOProcess::(_processLogs) (597) Queuing response >Thread-25::DEBUG::2014-10-15 07:01:57,005::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:01:57,011::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000412135 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:01:57,012::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:57,012::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:57,012::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:57,012::__init__::232::IOProcess::(_processLogs) (598) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:01:57,012::__init__::232::IOProcess::(_processLogs) (598) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:57,013::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:57,013::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:57,013::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:57,013::__init__::232::IOProcess::(_processLogs) (599) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:57,013::__init__::232::IOProcess::(_processLogs) (599) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:57,014::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:57,014::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:57,014::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:57,014::__init__::232::IOProcess::(_processLogs) (600) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:57,014::__init__::232::IOProcess::(_processLogs) (600) Queuing response >Thread-16::DEBUG::2014-10-15 07:01:57,015::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:01:57,015::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:01:57,015::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:01:57,015::__init__::232::IOProcess::(_processLogs) (601) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:01:57,015::__init__::232::IOProcess::(_processLogs) (601) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:01:57,713::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:01:58,714::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:58,723::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:58,724::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:01:58,724::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:59,608::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:59,609::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-728::DEBUG::2014-10-15 07:01:59,610::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:01:59,611::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:01:59,612::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-729::DEBUG::2014-10-15 07:01:59,612::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-729::DEBUG::2014-10-15 07:01:59,613::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:01:59,715::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:00,716::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:01,717::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:02,634::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:02,635::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-730::DEBUG::2014-10-15 07:02:02,636::task::595::Storage.TaskManager.Task::(_updateState) Task=`beb7c0b4-d058-4ee1-b699-04f9a8a75fc3`::moving from state init -> state preparing >Thread-730::INFO::2014-10-15 07:02:02,636::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-730::INFO::2014-10-15 07:02:02,636::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000412135', 'lastCheck': '5.6', 'valid': True}} >Thread-730::DEBUG::2014-10-15 07:02:02,636::task::1191::Storage.TaskManager.Task::(prepare) Task=`beb7c0b4-d058-4ee1-b699-04f9a8a75fc3`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000412135', 'lastCheck': '5.6', 'valid': True}} >Thread-730::DEBUG::2014-10-15 07:02:02,636::task::595::Storage.TaskManager.Task::(_updateState) Task=`beb7c0b4-d058-4ee1-b699-04f9a8a75fc3`::moving from state preparing -> state finished >Thread-730::DEBUG::2014-10-15 07:02:02,637::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-730::DEBUG::2014-10-15 07:02:02,637::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-730::DEBUG::2014-10-15 07:02:02,637::task::993::Storage.TaskManager.Task::(_decref) Task=`beb7c0b4-d058-4ee1-b699-04f9a8a75fc3`::ref 0 aborting False >Thread-730::ERROR::2014-10-15 07:02:02,637::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-730::DEBUG::2014-10-15 07:02:02,638::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:02,646::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:02,647::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-731::DEBUG::2014-10-15 07:02:02,648::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-731::DEBUG::2014-10-15 07:02:02,648::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:02,719::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:03,720::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:03,730::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:03,730::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:03,731::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-17::DEBUG::2014-10-15 07:02:03,734::task::595::Storage.TaskManager.Task::(_updateState) Task=`f74710f6-2ea4-4aec-89ea-8cc9ed7941be`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 07:02:03,735::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:02:03,735::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:03,735::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:03,735::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:03,735::__init__::232::IOProcess::(_processLogs) (602) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:02:03,737::__init__::232::IOProcess::(_processLogs) (602) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:03,737::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:03,737::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:03,737::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:03,738::__init__::232::IOProcess::(_processLogs) (603) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:02:03,738::__init__::232::IOProcess::(_processLogs) (603) Queuing response >Thread-17::INFO::2014-10-15 07:02:03,738::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:02:03,738::task::1191::Storage.TaskManager.Task::(prepare) Task=`f74710f6-2ea4-4aec-89ea-8cc9ed7941be`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:02:03,738::task::595::Storage.TaskManager.Task::(_updateState) Task=`f74710f6-2ea4-4aec-89ea-8cc9ed7941be`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 07:02:03,738::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 07:02:03,738::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 07:02:03,739::task::993::Storage.TaskManager.Task::(_decref) Task=`f74710f6-2ea4-4aec-89ea-8cc9ed7941be`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 07:02:04,722::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:05,723::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:05,825::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:05,826::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-732::DEBUG::2014-10-15 07:02:05,829::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:05,830::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:05,832::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-733::DEBUG::2014-10-15 07:02:05,833::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-733::DEBUG::2014-10-15 07:02:05,833::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:06,724::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:02:07,016::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:07,016::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:07,016::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:07,016::__init__::232::IOProcess::(_processLogs) (604) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:02:07,016::__init__::232::IOProcess::(_processLogs) (604) Queuing response >Thread-25::DEBUG::2014-10-15 07:02:07,017::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:02:07,024::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000428092 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:02:07,024::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:07,024::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:07,025::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:07,025::__init__::232::IOProcess::(_processLogs) (605) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:02:07,025::__init__::232::IOProcess::(_processLogs) (605) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:07,025::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:07,026::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:07,026::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:07,026::__init__::232::IOProcess::(_processLogs) (606) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:07,026::__init__::232::IOProcess::(_processLogs) (606) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:07,026::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:07,026::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:07,026::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:07,027::__init__::232::IOProcess::(_processLogs) (607) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:07,027::__init__::232::IOProcess::(_processLogs) (607) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:07,027::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:07,027::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:07,027::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:07,027::__init__::232::IOProcess::(_processLogs) (608) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:07,027::__init__::232::IOProcess::(_processLogs) (608) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:02:07,726::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:08,727::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:08,727::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:02:08,727::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:08,737::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:08,737::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:08,737::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:08,860::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:08,860::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-734::DEBUG::2014-10-15 07:02:08,861::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:08,863::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:08,863::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-735::DEBUG::2014-10-15 07:02:08,864::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-735::DEBUG::2014-10-15 07:02:08,864::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:09,729::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:10,730::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:11,731::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:11,886::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:11,887::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-736::DEBUG::2014-10-15 07:02:11,888::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:11,889::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:11,889::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-737::DEBUG::2014-10-15 07:02:11,890::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-737::DEBUG::2014-10-15 07:02:11,891::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:12,732::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:13,734::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:13,743::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:13,744::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:13,744::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:02:14,735::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:14,916::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:14,917::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-738::DEBUG::2014-10-15 07:02:14,918::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:14,919::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:14,919::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-739::DEBUG::2014-10-15 07:02:14,920::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-739::DEBUG::2014-10-15 07:02:14,921::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:15,736::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:16,737::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:02:17,028::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:17,028::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:17,029::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:17,029::__init__::232::IOProcess::(_processLogs) (609) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:02:17,029::__init__::232::IOProcess::(_processLogs) (609) Queuing response >Thread-25::DEBUG::2014-10-15 07:02:17,029::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:02:17,035::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000399167 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:02:17,036::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:17,036::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:17,036::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:17,037::__init__::232::IOProcess::(_processLogs) (610) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:02:17,037::__init__::232::IOProcess::(_processLogs) (610) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:17,037::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:17,037::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:17,037::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:17,038::__init__::232::IOProcess::(_processLogs) (611) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:17,038::__init__::232::IOProcess::(_processLogs) (611) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:17,038::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:17,038::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:17,038::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:17,038::__init__::232::IOProcess::(_processLogs) (612) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:17,038::__init__::232::IOProcess::(_processLogs) (612) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:17,039::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:17,039::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:17,039::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:17,039::__init__::232::IOProcess::(_processLogs) (613) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:17,039::__init__::232::IOProcess::(_processLogs) (613) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:02:17,739::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:17,942::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:17,942::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-740::DEBUG::2014-10-15 07:02:17,943::task::595::Storage.TaskManager.Task::(_updateState) Task=`198517b6-78db-41dd-94f3-08f817dba857`::moving from state init -> state preparing >Thread-740::INFO::2014-10-15 07:02:17,944::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-740::INFO::2014-10-15 07:02:17,944::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000399167', 'lastCheck': '0.9', 'valid': True}} >Thread-740::DEBUG::2014-10-15 07:02:17,944::task::1191::Storage.TaskManager.Task::(prepare) Task=`198517b6-78db-41dd-94f3-08f817dba857`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000399167', 'lastCheck': '0.9', 'valid': True}} >Thread-740::DEBUG::2014-10-15 07:02:17,944::task::595::Storage.TaskManager.Task::(_updateState) Task=`198517b6-78db-41dd-94f3-08f817dba857`::moving from state preparing -> state finished >Thread-740::DEBUG::2014-10-15 07:02:17,944::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-740::DEBUG::2014-10-15 07:02:17,944::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-740::DEBUG::2014-10-15 07:02:17,944::task::993::Storage.TaskManager.Task::(_decref) Task=`198517b6-78db-41dd-94f3-08f817dba857`::ref 0 aborting False >Thread-740::ERROR::2014-10-15 07:02:17,945::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-740::DEBUG::2014-10-15 07:02:17,946::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:17,956::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:17,957::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-741::DEBUG::2014-10-15 07:02:17,958::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-741::DEBUG::2014-10-15 07:02:17,958::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:18,740::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:18,750::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:18,751::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:18,751::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:02:19,741::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:20,743::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:21,066::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:21,067::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-742::DEBUG::2014-10-15 07:02:21,068::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:21,069::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:21,070::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-743::DEBUG::2014-10-15 07:02:21,071::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-743::DEBUG::2014-10-15 07:02:21,071::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:21,744::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:22,745::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:23,746::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:23,757::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:23,757::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:23,758::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:24,103::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:24,103::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-744::DEBUG::2014-10-15 07:02:24,104::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:24,105::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:24,106::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-745::DEBUG::2014-10-15 07:02:24,107::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-745::DEBUG::2014-10-15 07:02:24,107::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:24,747::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:25,748::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:26,749::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:02:27,040::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:27,040::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:27,040::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:27,040::__init__::232::IOProcess::(_processLogs) (614) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:02:27,041::__init__::232::IOProcess::(_processLogs) (614) Queuing response >Thread-25::DEBUG::2014-10-15 07:02:27,041::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:02:27,047::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000391164 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:02:27,048::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:27,048::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:27,048::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:27,048::__init__::232::IOProcess::(_processLogs) (615) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:02:27,048::__init__::232::IOProcess::(_processLogs) (615) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:27,049::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:27,049::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:27,049::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:27,049::__init__::232::IOProcess::(_processLogs) (616) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:27,049::__init__::232::IOProcess::(_processLogs) (616) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:27,049::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:27,050::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:27,050::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:27,050::__init__::232::IOProcess::(_processLogs) (617) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:27,050::__init__::232::IOProcess::(_processLogs) (617) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:27,050::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:27,050::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:27,050::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:27,050::__init__::232::IOProcess::(_processLogs) (618) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:27,051::__init__::232::IOProcess::(_processLogs) (618) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:27,121::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:27,121::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-746::DEBUG::2014-10-15 07:02:27,122::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:27,123::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:27,124::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-747::DEBUG::2014-10-15 07:02:27,125::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-747::DEBUG::2014-10-15 07:02:27,125::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:27,751::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:28,752::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:28,764::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:28,764::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:28,765::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:02:29,753::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:30,148::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:30,148::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-748::DEBUG::2014-10-15 07:02:30,149::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:30,151::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:30,151::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-749::DEBUG::2014-10-15 07:02:30,152::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-749::DEBUG::2014-10-15 07:02:30,152::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:30,754::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:31,756::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:32,757::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:33,177::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:33,178::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-750::DEBUG::2014-10-15 07:02:33,179::task::595::Storage.TaskManager.Task::(_updateState) Task=`aac20df8-f228-478b-ba55-97ed12ce78fa`::moving from state init -> state preparing >Thread-750::INFO::2014-10-15 07:02:33,179::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-750::INFO::2014-10-15 07:02:33,180::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000391164', 'lastCheck': '6.1', 'valid': True}} >Thread-750::DEBUG::2014-10-15 07:02:33,180::task::1191::Storage.TaskManager.Task::(prepare) Task=`aac20df8-f228-478b-ba55-97ed12ce78fa`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000391164', 'lastCheck': '6.1', 'valid': True}} >Thread-750::DEBUG::2014-10-15 07:02:33,180::task::595::Storage.TaskManager.Task::(_updateState) Task=`aac20df8-f228-478b-ba55-97ed12ce78fa`::moving from state preparing -> state finished >Thread-750::DEBUG::2014-10-15 07:02:33,180::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-750::DEBUG::2014-10-15 07:02:33,180::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-750::DEBUG::2014-10-15 07:02:33,180::task::993::Storage.TaskManager.Task::(_decref) Task=`aac20df8-f228-478b-ba55-97ed12ce78fa`::ref 0 aborting False >Thread-750::ERROR::2014-10-15 07:02:33,181::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-750::DEBUG::2014-10-15 07:02:33,182::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:33,194::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:33,195::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-751::DEBUG::2014-10-15 07:02:33,196::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-751::DEBUG::2014-10-15 07:02:33,196::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:33,759::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:33,770::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:33,771::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:33,771::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:02:34,760::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:35,761::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:36,303::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:36,304::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-752::DEBUG::2014-10-15 07:02:36,304::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:36,306::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:36,306::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-753::DEBUG::2014-10-15 07:02:36,307::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-753::DEBUG::2014-10-15 07:02:36,307::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:36,762::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:02:37,051::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:37,052::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:37,052::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:37,052::__init__::232::IOProcess::(_processLogs) (619) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:02:37,052::__init__::232::IOProcess::(_processLogs) (619) Queuing response >Thread-25::DEBUG::2014-10-15 07:02:37,052::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:02:37,058::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000386921 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:02:37,059::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:37,059::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:37,059::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:37,060::__init__::232::IOProcess::(_processLogs) (620) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:02:37,060::__init__::232::IOProcess::(_processLogs) (620) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:37,060::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:37,061::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:37,061::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:37,061::__init__::232::IOProcess::(_processLogs) (621) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:37,061::__init__::232::IOProcess::(_processLogs) (621) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:37,061::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:37,061::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:37,062::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:37,062::__init__::232::IOProcess::(_processLogs) (622) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:37,062::__init__::232::IOProcess::(_processLogs) (622) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:37,062::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:37,062::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:37,062::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:37,062::__init__::232::IOProcess::(_processLogs) (623) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:37,062::__init__::232::IOProcess::(_processLogs) (623) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:02:37,764::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:38,765::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:38,765::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:02:38,765::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:38,777::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:38,778::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:38,778::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:39,342::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:39,343::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-754::DEBUG::2014-10-15 07:02:39,344::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:39,345::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:39,346::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-755::DEBUG::2014-10-15 07:02:39,346::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-755::DEBUG::2014-10-15 07:02:39,347::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:39,766::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:40,767::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:41,768::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:42,370::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:42,371::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-756::DEBUG::2014-10-15 07:02:42,372::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:42,373::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:42,373::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-757::DEBUG::2014-10-15 07:02:42,374::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-757::DEBUG::2014-10-15 07:02:42,375::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:42,769::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:43,770::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:43,784::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:43,784::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:43,785::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:02:44,771::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:45,388::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:45,389::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-758::DEBUG::2014-10-15 07:02:45,390::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:45,391::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:45,391::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-759::DEBUG::2014-10-15 07:02:45,392::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-759::DEBUG::2014-10-15 07:02:45,392::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:45,773::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:46,774::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:02:47,063::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:47,064::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:47,064::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:47,064::__init__::232::IOProcess::(_processLogs) (624) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:02:47,064::__init__::232::IOProcess::(_processLogs) (624) Queuing response >Thread-25::DEBUG::2014-10-15 07:02:47,064::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:02:47,070::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000401009 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:02:47,071::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:47,071::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:47,071::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:47,071::__init__::232::IOProcess::(_processLogs) (625) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:02:47,071::__init__::232::IOProcess::(_processLogs) (625) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:47,072::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:47,072::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:47,072::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:47,072::__init__::232::IOProcess::(_processLogs) (626) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:47,072::__init__::232::IOProcess::(_processLogs) (626) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:47,073::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:47,073::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:47,073::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:47,073::__init__::232::IOProcess::(_processLogs) (627) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:47,073::__init__::232::IOProcess::(_processLogs) (627) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:47,074::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:47,074::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:47,074::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:47,074::__init__::232::IOProcess::(_processLogs) (628) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:47,074::__init__::232::IOProcess::(_processLogs) (628) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:02:47,775::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:48,408::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:48,409::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-760::DEBUG::2014-10-15 07:02:48,411::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d142cc8-388b-4ed6-8ef6-29d0e8d06ac7`::moving from state init -> state preparing >Thread-760::INFO::2014-10-15 07:02:48,411::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-760::INFO::2014-10-15 07:02:48,411::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000401009', 'lastCheck': '1.3', 'valid': True}} >Thread-760::DEBUG::2014-10-15 07:02:48,411::task::1191::Storage.TaskManager.Task::(prepare) Task=`2d142cc8-388b-4ed6-8ef6-29d0e8d06ac7`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000401009', 'lastCheck': '1.3', 'valid': True}} >Thread-760::DEBUG::2014-10-15 07:02:48,411::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d142cc8-388b-4ed6-8ef6-29d0e8d06ac7`::moving from state preparing -> state finished >Thread-760::DEBUG::2014-10-15 07:02:48,411::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-760::DEBUG::2014-10-15 07:02:48,411::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-760::DEBUG::2014-10-15 07:02:48,412::task::993::Storage.TaskManager.Task::(_decref) Task=`2d142cc8-388b-4ed6-8ef6-29d0e8d06ac7`::ref 0 aborting False >Thread-760::ERROR::2014-10-15 07:02:48,412::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-760::DEBUG::2014-10-15 07:02:48,413::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:48,425::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:48,426::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-761::DEBUG::2014-10-15 07:02:48,427::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-761::DEBUG::2014-10-15 07:02:48,427::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:48,776::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:48,790::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:48,791::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:48,791::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:02:49,777::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:50,779::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:51,531::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:51,532::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-762::DEBUG::2014-10-15 07:02:51,533::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:51,534::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:51,534::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-763::DEBUG::2014-10-15 07:02:51,535::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-763::DEBUG::2014-10-15 07:02:51,536::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:51,780::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:52,781::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:53,782::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:53,797::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:53,798::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:53,798::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:54,565::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:54,565::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-764::DEBUG::2014-10-15 07:02:54,566::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:54,567::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:54,568::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-765::DEBUG::2014-10-15 07:02:54,569::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-765::DEBUG::2014-10-15 07:02:54,569::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:54,784::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:55,785::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:56,786::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:02:57,075::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:57,075::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:57,075::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:57,075::__init__::232::IOProcess::(_processLogs) (629) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:02:57,076::__init__::232::IOProcess::(_processLogs) (629) Queuing response >Thread-25::DEBUG::2014-10-15 07:02:57,076::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:02:57,082::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000375541 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:02:57,083::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:57,083::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:57,083::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:57,083::__init__::232::IOProcess::(_processLogs) (630) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:02:57,083::__init__::232::IOProcess::(_processLogs) (630) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:57,084::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:57,084::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:57,084::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:57,084::__init__::232::IOProcess::(_processLogs) (631) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:57,084::__init__::232::IOProcess::(_processLogs) (631) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:57,085::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:57,085::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:57,085::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:57,085::__init__::232::IOProcess::(_processLogs) (632) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:57,085::__init__::232::IOProcess::(_processLogs) (632) Queuing response >Thread-16::DEBUG::2014-10-15 07:02:57,085::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:02:57,085::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:02:57,086::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:02:57,086::__init__::232::IOProcess::(_processLogs) (633) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:02:57,086::__init__::232::IOProcess::(_processLogs) (633) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:57,593::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:57,594::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-766::DEBUG::2014-10-15 07:02:57,595::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:02:57,596::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:02:57,597::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-767::DEBUG::2014-10-15 07:02:57,598::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-767::DEBUG::2014-10-15 07:02:57,598::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:02:57,788::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:02:58,789::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:58,800::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:58,800::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:02:58,801::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:02:59,790::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:00,617::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:00,618::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-768::DEBUG::2014-10-15 07:03:00,619::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:00,620::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:00,620::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-769::DEBUG::2014-10-15 07:03:00,621::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-769::DEBUG::2014-10-15 07:03:00,622::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:00,791::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:01,793::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:02,794::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:03,635::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:03,635::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-770::DEBUG::2014-10-15 07:03:03,637::task::595::Storage.TaskManager.Task::(_updateState) Task=`f821442c-7215-4a88-a3aa-03587bfd1871`::moving from state init -> state preparing >Thread-770::INFO::2014-10-15 07:03:03,637::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-770::INFO::2014-10-15 07:03:03,637::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000375541', 'lastCheck': '6.6', 'valid': True}} >Thread-770::DEBUG::2014-10-15 07:03:03,637::task::1191::Storage.TaskManager.Task::(prepare) Task=`f821442c-7215-4a88-a3aa-03587bfd1871`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000375541', 'lastCheck': '6.6', 'valid': True}} >Thread-770::DEBUG::2014-10-15 07:03:03,637::task::595::Storage.TaskManager.Task::(_updateState) Task=`f821442c-7215-4a88-a3aa-03587bfd1871`::moving from state preparing -> state finished >Thread-770::DEBUG::2014-10-15 07:03:03,637::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-770::DEBUG::2014-10-15 07:03:03,638::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-770::DEBUG::2014-10-15 07:03:03,638::task::993::Storage.TaskManager.Task::(_decref) Task=`f821442c-7215-4a88-a3aa-03587bfd1871`::ref 0 aborting False >Thread-770::ERROR::2014-10-15 07:03:03,639::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-770::DEBUG::2014-10-15 07:03:03,639::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:03,646::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:03,646::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-771::DEBUG::2014-10-15 07:03:03,647::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-771::DEBUG::2014-10-15 07:03:03,648::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-17::DEBUG::2014-10-15 07:03:03,776::task::595::Storage.TaskManager.Task::(_updateState) Task=`2aa7105d-02f1-4aad-8b45-c0741d15b95b`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 07:03:03,776::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:03:03,776::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:03,776::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:03,776::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:03,777::__init__::232::IOProcess::(_processLogs) (634) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:03:03,778::__init__::232::IOProcess::(_processLogs) (634) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:03,778::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:03,779::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:03,779::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:03,779::__init__::232::IOProcess::(_processLogs) (635) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:03:03,779::__init__::232::IOProcess::(_processLogs) (635) Queuing response >Thread-17::INFO::2014-10-15 07:03:03,779::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:03:03,779::task::1191::Storage.TaskManager.Task::(prepare) Task=`2aa7105d-02f1-4aad-8b45-c0741d15b95b`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:03:03,779::task::595::Storage.TaskManager.Task::(_updateState) Task=`2aa7105d-02f1-4aad-8b45-c0741d15b95b`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 07:03:03,779::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 07:03:03,780::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 07:03:03,780::task::993::Storage.TaskManager.Task::(_decref) Task=`2aa7105d-02f1-4aad-8b45-c0741d15b95b`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 07:03:03,794::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:03,805::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:03,806::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:03,806::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:03:04,796::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:05,797::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:06,775::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:06,776::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-772::DEBUG::2014-10-15 07:03:06,777::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:06,778::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:06,779::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-773::DEBUG::2014-10-15 07:03:06,780::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-773::DEBUG::2014-10-15 07:03:06,780::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:06,798::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:03:07,087::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:07,087::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:07,087::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:07,087::__init__::232::IOProcess::(_processLogs) (636) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:03:07,087::__init__::232::IOProcess::(_processLogs) (636) Queuing response >Thread-25::DEBUG::2014-10-15 07:03:07,087::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:03:07,094::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000380944 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:03:07,095::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:07,095::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:07,095::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:07,095::__init__::232::IOProcess::(_processLogs) (637) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:03:07,095::__init__::232::IOProcess::(_processLogs) (637) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:07,096::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:07,096::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:07,096::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:07,096::__init__::232::IOProcess::(_processLogs) (638) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:07,096::__init__::232::IOProcess::(_processLogs) (638) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:07,097::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:07,097::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:07,097::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:07,097::__init__::232::IOProcess::(_processLogs) (639) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:07,097::__init__::232::IOProcess::(_processLogs) (639) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:07,097::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:07,097::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:07,097::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:07,098::__init__::232::IOProcess::(_processLogs) (640) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:07,098::__init__::232::IOProcess::(_processLogs) (640) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:03:07,799::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:08,801::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:08,801::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:03:08,801::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:08,808::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:08,808::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:08,809::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:03:09,802::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:09,808::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:09,808::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-774::DEBUG::2014-10-15 07:03:09,809::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:09,810::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:09,811::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-775::DEBUG::2014-10-15 07:03:09,812::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-775::DEBUG::2014-10-15 07:03:09,812::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:10,804::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:11,805::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:12,806::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:12,860::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:12,860::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-776::DEBUG::2014-10-15 07:03:12,861::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:12,862::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:12,863::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-777::DEBUG::2014-10-15 07:03:12,863::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-777::DEBUG::2014-10-15 07:03:12,864::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:13,808::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:13,815::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:13,815::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:13,816::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:03:14,808::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:15,810::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:15,878::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:15,879::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-778::DEBUG::2014-10-15 07:03:15,880::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:15,881::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:15,882::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-779::DEBUG::2014-10-15 07:03:15,883::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-779::DEBUG::2014-10-15 07:03:15,883::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:16,811::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:03:17,099::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:17,099::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:17,099::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:17,099::__init__::232::IOProcess::(_processLogs) (641) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:03:17,099::__init__::232::IOProcess::(_processLogs) (641) Queuing response >Thread-25::DEBUG::2014-10-15 07:03:17,100::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:03:17,106::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000365165 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:03:17,106::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:17,106::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:17,107::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:17,107::__init__::232::IOProcess::(_processLogs) (642) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:03:17,107::__init__::232::IOProcess::(_processLogs) (642) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:17,107::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:17,108::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:17,108::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:17,108::__init__::232::IOProcess::(_processLogs) (643) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:17,108::__init__::232::IOProcess::(_processLogs) (643) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:17,108::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:17,108::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:17,108::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:17,108::__init__::232::IOProcess::(_processLogs) (644) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:17,109::__init__::232::IOProcess::(_processLogs) (644) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:17,109::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:17,109::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:17,109::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:17,109::__init__::232::IOProcess::(_processLogs) (645) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:17,109::__init__::232::IOProcess::(_processLogs) (645) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:03:17,812::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:18,814::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:18,821::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:18,822::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:18,822::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:18,925::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:18,926::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-780::DEBUG::2014-10-15 07:03:18,927::task::595::Storage.TaskManager.Task::(_updateState) Task=`160b5090-d878-4b8b-931c-0f1b0990c52e`::moving from state init -> state preparing >Thread-780::INFO::2014-10-15 07:03:18,927::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-780::INFO::2014-10-15 07:03:18,927::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000365165', 'lastCheck': '1.8', 'valid': True}} >Thread-780::DEBUG::2014-10-15 07:03:18,927::task::1191::Storage.TaskManager.Task::(prepare) Task=`160b5090-d878-4b8b-931c-0f1b0990c52e`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000365165', 'lastCheck': '1.8', 'valid': True}} >Thread-780::DEBUG::2014-10-15 07:03:18,927::task::595::Storage.TaskManager.Task::(_updateState) Task=`160b5090-d878-4b8b-931c-0f1b0990c52e`::moving from state preparing -> state finished >Thread-780::DEBUG::2014-10-15 07:03:18,928::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-780::DEBUG::2014-10-15 07:03:18,928::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-780::DEBUG::2014-10-15 07:03:18,928::task::993::Storage.TaskManager.Task::(_decref) Task=`160b5090-d878-4b8b-931c-0f1b0990c52e`::ref 0 aborting False >Thread-780::ERROR::2014-10-15 07:03:18,929::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-780::DEBUG::2014-10-15 07:03:18,929::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:18,936::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:18,937::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-781::DEBUG::2014-10-15 07:03:18,938::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-781::DEBUG::2014-10-15 07:03:18,938::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:19,814::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:20,816::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:21,817::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:22,047::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:22,047::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-782::DEBUG::2014-10-15 07:03:22,048::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:22,049::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:22,050::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-783::DEBUG::2014-10-15 07:03:22,051::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-783::DEBUG::2014-10-15 07:03:22,051::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:22,818::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:23,819::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:23,828::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:23,829::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:23,829::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:03:24,820::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:25,090::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:25,090::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-784::DEBUG::2014-10-15 07:03:25,092::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:25,093::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:25,093::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-785::DEBUG::2014-10-15 07:03:25,094::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-785::DEBUG::2014-10-15 07:03:25,094::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:25,821::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:26,822::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:03:27,110::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:27,110::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:27,110::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:27,111::__init__::232::IOProcess::(_processLogs) (646) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:03:27,111::__init__::232::IOProcess::(_processLogs) (646) Queuing response >Thread-25::DEBUG::2014-10-15 07:03:27,111::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:03:27,117::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00041824 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:03:27,118::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:27,118::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:27,118::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:27,118::__init__::232::IOProcess::(_processLogs) (647) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:03:27,118::__init__::232::IOProcess::(_processLogs) (647) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:27,119::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:27,119::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:27,119::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:27,119::__init__::232::IOProcess::(_processLogs) (648) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:27,119::__init__::232::IOProcess::(_processLogs) (648) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:27,120::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:27,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:27,120::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:27,120::__init__::232::IOProcess::(_processLogs) (649) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:27,120::__init__::232::IOProcess::(_processLogs) (649) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:27,120::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:27,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:27,120::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:27,121::__init__::232::IOProcess::(_processLogs) (650) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:27,121::__init__::232::IOProcess::(_processLogs) (650) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:03:27,824::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:28,112::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:28,113::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-786::DEBUG::2014-10-15 07:03:28,114::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:28,115::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:28,115::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-787::DEBUG::2014-10-15 07:03:28,116::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-787::DEBUG::2014-10-15 07:03:28,117::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:28,825::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:28,832::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:28,832::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:28,833::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:03:29,826::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:30,828::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:31,137::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:31,138::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-788::DEBUG::2014-10-15 07:03:31,139::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:31,140::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:31,141::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-789::DEBUG::2014-10-15 07:03:31,142::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-789::DEBUG::2014-10-15 07:03:31,142::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:31,829::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:32,830::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:33,832::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:33,838::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:33,839::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:33,839::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:34,166::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:34,167::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-790::DEBUG::2014-10-15 07:03:34,168::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2d0aa87-26af-4d05-a28d-00e1f422e129`::moving from state init -> state preparing >Thread-790::INFO::2014-10-15 07:03:34,168::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-790::INFO::2014-10-15 07:03:34,169::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041824', 'lastCheck': '7.0', 'valid': True}} >Thread-790::DEBUG::2014-10-15 07:03:34,169::task::1191::Storage.TaskManager.Task::(prepare) Task=`e2d0aa87-26af-4d05-a28d-00e1f422e129`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041824', 'lastCheck': '7.0', 'valid': True}} >Thread-790::DEBUG::2014-10-15 07:03:34,169::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2d0aa87-26af-4d05-a28d-00e1f422e129`::moving from state preparing -> state finished >Thread-790::DEBUG::2014-10-15 07:03:34,169::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-790::DEBUG::2014-10-15 07:03:34,169::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-790::DEBUG::2014-10-15 07:03:34,169::task::993::Storage.TaskManager.Task::(_decref) Task=`e2d0aa87-26af-4d05-a28d-00e1f422e129`::ref 0 aborting False >Thread-790::ERROR::2014-10-15 07:03:34,170::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-790::DEBUG::2014-10-15 07:03:34,171::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:34,182::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:34,182::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-791::DEBUG::2014-10-15 07:03:34,183::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-791::DEBUG::2014-10-15 07:03:34,183::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:34,833::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:35,834::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:36,835::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:03:37,122::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:37,122::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:37,122::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:37,122::__init__::232::IOProcess::(_processLogs) (651) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:03:37,122::__init__::232::IOProcess::(_processLogs) (651) Queuing response >Thread-25::DEBUG::2014-10-15 07:03:37,123::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:03:37,129::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000378912 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:03:37,129::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:37,130::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:37,130::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:37,130::__init__::232::IOProcess::(_processLogs) (652) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:03:37,130::__init__::232::IOProcess::(_processLogs) (652) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:37,130::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:37,130::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:37,131::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:37,131::__init__::232::IOProcess::(_processLogs) (653) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:37,131::__init__::232::IOProcess::(_processLogs) (653) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:37,131::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:37,131::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:37,131::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:37,131::__init__::232::IOProcess::(_processLogs) (654) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:37,132::__init__::232::IOProcess::(_processLogs) (654) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:37,132::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:37,132::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:37,132::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:37,132::__init__::232::IOProcess::(_processLogs) (655) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:37,132::__init__::232::IOProcess::(_processLogs) (655) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:37,279::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:37,280::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-792::DEBUG::2014-10-15 07:03:37,281::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:37,282::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:37,282::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-793::DEBUG::2014-10-15 07:03:37,283::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-793::DEBUG::2014-10-15 07:03:37,283::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:37,837::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:38,838::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:38,838::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:03:38,838::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:38,845::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:38,846::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:38,846::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:03:39,839::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:40,316::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:40,316::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-794::DEBUG::2014-10-15 07:03:40,317::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:40,318::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:40,319::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-795::DEBUG::2014-10-15 07:03:40,320::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-795::DEBUG::2014-10-15 07:03:40,320::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:40,840::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:41,842::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:42,843::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:43,362::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:43,363::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-796::DEBUG::2014-10-15 07:03:43,363::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:43,365::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:43,365::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-797::DEBUG::2014-10-15 07:03:43,366::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-797::DEBUG::2014-10-15 07:03:43,367::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:43,844::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:43,852::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:43,852::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:43,853::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:03:44,845::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:45,847::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:46,388::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:46,389::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-798::DEBUG::2014-10-15 07:03:46,390::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:46,391::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:46,391::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-799::DEBUG::2014-10-15 07:03:46,392::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-799::DEBUG::2014-10-15 07:03:46,393::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:46,848::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:03:47,133::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:47,133::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:47,134::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:47,134::__init__::232::IOProcess::(_processLogs) (656) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:03:47,134::__init__::232::IOProcess::(_processLogs) (656) Queuing response >Thread-25::DEBUG::2014-10-15 07:03:47,134::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:03:47,140::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000445105 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:03:47,141::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:47,141::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:47,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:47,141::__init__::232::IOProcess::(_processLogs) (657) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:03:47,142::__init__::232::IOProcess::(_processLogs) (657) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:47,142::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:47,142::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:47,142::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:47,143::__init__::232::IOProcess::(_processLogs) (658) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:47,143::__init__::232::IOProcess::(_processLogs) (658) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:47,143::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:47,143::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:47,143::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:47,143::__init__::232::IOProcess::(_processLogs) (659) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:47,143::__init__::232::IOProcess::(_processLogs) (659) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:47,144::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:47,144::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:47,144::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:47,144::__init__::232::IOProcess::(_processLogs) (660) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:47,144::__init__::232::IOProcess::(_processLogs) (660) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:03:47,849::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:48,850::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:48,858::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:48,858::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:48,859::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:49,412::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:49,413::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-800::DEBUG::2014-10-15 07:03:49,414::task::595::Storage.TaskManager.Task::(_updateState) Task=`b32c0b02-a7d1-46b8-aa01-b68b389ae468`::moving from state init -> state preparing >Thread-800::INFO::2014-10-15 07:03:49,414::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-800::INFO::2014-10-15 07:03:49,415::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000445105', 'lastCheck': '2.3', 'valid': True}} >Thread-800::DEBUG::2014-10-15 07:03:49,415::task::1191::Storage.TaskManager.Task::(prepare) Task=`b32c0b02-a7d1-46b8-aa01-b68b389ae468`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000445105', 'lastCheck': '2.3', 'valid': True}} >Thread-800::DEBUG::2014-10-15 07:03:49,415::task::595::Storage.TaskManager.Task::(_updateState) Task=`b32c0b02-a7d1-46b8-aa01-b68b389ae468`::moving from state preparing -> state finished >Thread-800::DEBUG::2014-10-15 07:03:49,415::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-800::DEBUG::2014-10-15 07:03:49,415::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-800::DEBUG::2014-10-15 07:03:49,415::task::993::Storage.TaskManager.Task::(_decref) Task=`b32c0b02-a7d1-46b8-aa01-b68b389ae468`::ref 0 aborting False >Thread-800::ERROR::2014-10-15 07:03:49,416::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-800::DEBUG::2014-10-15 07:03:49,417::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:49,425::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:49,426::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-801::DEBUG::2014-10-15 07:03:49,427::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-801::DEBUG::2014-10-15 07:03:49,427::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:49,852::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:50,853::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:51,854::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:52,561::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:52,561::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-802::DEBUG::2014-10-15 07:03:52,562::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:52,564::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:52,564::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-803::DEBUG::2014-10-15 07:03:52,565::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-803::DEBUG::2014-10-15 07:03:52,565::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:52,856::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:53,857::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:53,865::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:53,865::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:53,865::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:03:54,858::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:55,604::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:55,605::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-804::DEBUG::2014-10-15 07:03:55,606::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:55,607::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:55,607::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-805::DEBUG::2014-10-15 07:03:55,608::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-805::DEBUG::2014-10-15 07:03:55,609::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:55,859::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:03:56,861::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:03:57,145::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:57,145::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:57,145::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:57,145::__init__::232::IOProcess::(_processLogs) (661) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:03:57,146::__init__::232::IOProcess::(_processLogs) (661) Queuing response >Thread-25::DEBUG::2014-10-15 07:03:57,146::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:03:57,152::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000349941 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:03:57,153::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:57,153::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:57,153::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:57,153::__init__::232::IOProcess::(_processLogs) (662) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:03:57,153::__init__::232::IOProcess::(_processLogs) (662) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:57,154::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:57,154::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:57,154::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:57,154::__init__::232::IOProcess::(_processLogs) (663) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:57,154::__init__::232::IOProcess::(_processLogs) (663) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:57,154::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:57,154::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:57,155::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:57,155::__init__::232::IOProcess::(_processLogs) (664) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:57,155::__init__::232::IOProcess::(_processLogs) (664) Queuing response >Thread-16::DEBUG::2014-10-15 07:03:57,155::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:03:57,155::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:03:57,155::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:03:57,155::__init__::232::IOProcess::(_processLogs) (665) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:03:57,155::__init__::232::IOProcess::(_processLogs) (665) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:03:57,862::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:58,623::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:58,624::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-806::DEBUG::2014-10-15 07:03:58,625::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:03:58,626::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:03:58,627::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-807::DEBUG::2014-10-15 07:03:58,628::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-807::DEBUG::2014-10-15 07:03:58,628::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:03:58,863::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:58,871::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:58,872::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:03:58,872::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:03:59,863::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:00,865::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:01,646::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:01,648::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-808::DEBUG::2014-10-15 07:04:01,650::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:01,652::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:01,654::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-809::DEBUG::2014-10-15 07:04:01,655::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-809::DEBUG::2014-10-15 07:04:01,657::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:01,866::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:02,868::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-17::DEBUG::2014-10-15 07:04:03,819::task::595::Storage.TaskManager.Task::(_updateState) Task=`c5728a4f-7a41-4977-8064-7b177798e7a2`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 07:04:03,819::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:04:03,820::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:03,820::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:03,820::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:03,820::__init__::232::IOProcess::(_processLogs) (666) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:04:03,821::__init__::232::IOProcess::(_processLogs) (666) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:03,821::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:03,822::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:03,822::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:03,822::__init__::232::IOProcess::(_processLogs) (667) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:04:03,822::__init__::232::IOProcess::(_processLogs) (667) Queuing response >Thread-17::INFO::2014-10-15 07:04:03,822::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:04:03,822::task::1191::Storage.TaskManager.Task::(prepare) Task=`c5728a4f-7a41-4977-8064-7b177798e7a2`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:04:03,822::task::595::Storage.TaskManager.Task::(_updateState) Task=`c5728a4f-7a41-4977-8064-7b177798e7a2`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 07:04:03,822::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 07:04:03,822::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 07:04:03,823::task::993::Storage.TaskManager.Task::(_decref) Task=`c5728a4f-7a41-4977-8064-7b177798e7a2`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 07:04:03,869::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:03,878::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:03,878::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:03,879::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:04,681::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:04,682::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-810::DEBUG::2014-10-15 07:04:04,683::task::595::Storage.TaskManager.Task::(_updateState) Task=`77880acc-63b3-4e5e-ae86-726cc4f1af69`::moving from state init -> state preparing >Thread-810::INFO::2014-10-15 07:04:04,683::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-810::INFO::2014-10-15 07:04:04,683::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000349941', 'lastCheck': '7.5', 'valid': True}} >Thread-810::DEBUG::2014-10-15 07:04:04,683::task::1191::Storage.TaskManager.Task::(prepare) Task=`77880acc-63b3-4e5e-ae86-726cc4f1af69`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000349941', 'lastCheck': '7.5', 'valid': True}} >Thread-810::DEBUG::2014-10-15 07:04:04,683::task::595::Storage.TaskManager.Task::(_updateState) Task=`77880acc-63b3-4e5e-ae86-726cc4f1af69`::moving from state preparing -> state finished >Thread-810::DEBUG::2014-10-15 07:04:04,684::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-810::DEBUG::2014-10-15 07:04:04,684::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-810::DEBUG::2014-10-15 07:04:04,684::task::993::Storage.TaskManager.Task::(_decref) Task=`77880acc-63b3-4e5e-ae86-726cc4f1af69`::ref 0 aborting False >Thread-810::ERROR::2014-10-15 07:04:04,685::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-810::DEBUG::2014-10-15 07:04:04,685::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:04,695::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:04,696::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-811::DEBUG::2014-10-15 07:04:04,697::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-811::DEBUG::2014-10-15 07:04:04,697::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:04,870::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:05,871::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:06,873::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:04:07,156::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:07,157::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:07,157::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:07,157::__init__::232::IOProcess::(_processLogs) (668) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:04:07,157::__init__::232::IOProcess::(_processLogs) (668) Queuing response >Thread-25::DEBUG::2014-10-15 07:04:07,157::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:04:07,164::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000385932 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:04:07,165::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:07,165::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:07,165::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:07,165::__init__::232::IOProcess::(_processLogs) (669) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:04:07,165::__init__::232::IOProcess::(_processLogs) (669) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:07,166::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:07,166::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:07,166::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:07,166::__init__::232::IOProcess::(_processLogs) (670) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:07,166::__init__::232::IOProcess::(_processLogs) (670) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:07,166::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:07,167::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:07,167::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:07,167::__init__::232::IOProcess::(_processLogs) (671) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:07,167::__init__::232::IOProcess::(_processLogs) (671) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:07,167::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:07,167::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:07,167::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:07,168::__init__::232::IOProcess::(_processLogs) (672) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:07,168::__init__::232::IOProcess::(_processLogs) (672) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:07,810::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:07,811::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-812::DEBUG::2014-10-15 07:04:07,812::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:07,813::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:07,814::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-813::DEBUG::2014-10-15 07:04:07,814::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-813::DEBUG::2014-10-15 07:04:07,815::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:07,873::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:08,875::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:08,875::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:04:08,875::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:08,884::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:08,885::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:08,885::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:04:09,876::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:10,840::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:10,841::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-814::DEBUG::2014-10-15 07:04:10,842::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:10,843::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:10,844::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-815::DEBUG::2014-10-15 07:04:10,845::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-815::DEBUG::2014-10-15 07:04:10,845::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:10,878::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:11,879::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:12,880::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:13,866::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:13,867::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-816::DEBUG::2014-10-15 07:04:13,868::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:13,870::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:13,870::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-817::DEBUG::2014-10-15 07:04:13,871::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-817::DEBUG::2014-10-15 07:04:13,871::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:13,881::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:13,891::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:13,891::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:13,892::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:04:14,882::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:15,884::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:16,885::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:16,886::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:16,886::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-818::DEBUG::2014-10-15 07:04:16,887::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:16,888::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:16,889::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-819::DEBUG::2014-10-15 07:04:16,889::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-819::DEBUG::2014-10-15 07:04:16,890::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:04:17,169::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:17,169::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:17,169::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:17,169::__init__::232::IOProcess::(_processLogs) (673) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:04:17,169::__init__::232::IOProcess::(_processLogs) (673) Queuing response >Thread-25::DEBUG::2014-10-15 07:04:17,169::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:04:17,175::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00036202 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:04:17,176::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:17,176::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:17,176::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:17,176::__init__::232::IOProcess::(_processLogs) (674) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:04:17,177::__init__::232::IOProcess::(_processLogs) (674) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:17,177::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:17,177::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:17,177::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:17,177::__init__::232::IOProcess::(_processLogs) (675) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:17,178::__init__::232::IOProcess::(_processLogs) (675) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:17,178::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:17,178::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:17,178::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:17,178::__init__::232::IOProcess::(_processLogs) (676) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:17,178::__init__::232::IOProcess::(_processLogs) (676) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:17,179::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:17,179::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:17,179::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:17,179::__init__::232::IOProcess::(_processLogs) (677) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:17,179::__init__::232::IOProcess::(_processLogs) (677) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:04:17,887::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:18,888::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:18,898::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:18,898::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:18,899::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:04:19,889::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:19,910::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:19,911::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-820::DEBUG::2014-10-15 07:04:19,912::task::595::Storage.TaskManager.Task::(_updateState) Task=`20d92a0b-5918-4eb7-95ae-a9ab1818d2af`::moving from state init -> state preparing >Thread-820::INFO::2014-10-15 07:04:19,912::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-820::INFO::2014-10-15 07:04:19,912::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00036202', 'lastCheck': '2.7', 'valid': True}} >Thread-820::DEBUG::2014-10-15 07:04:19,912::task::1191::Storage.TaskManager.Task::(prepare) Task=`20d92a0b-5918-4eb7-95ae-a9ab1818d2af`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00036202', 'lastCheck': '2.7', 'valid': True}} >Thread-820::DEBUG::2014-10-15 07:04:19,912::task::595::Storage.TaskManager.Task::(_updateState) Task=`20d92a0b-5918-4eb7-95ae-a9ab1818d2af`::moving from state preparing -> state finished >Thread-820::DEBUG::2014-10-15 07:04:19,913::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-820::DEBUG::2014-10-15 07:04:19,913::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-820::DEBUG::2014-10-15 07:04:19,913::task::993::Storage.TaskManager.Task::(_decref) Task=`20d92a0b-5918-4eb7-95ae-a9ab1818d2af`::ref 0 aborting False >Thread-820::ERROR::2014-10-15 07:04:19,914::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-820::DEBUG::2014-10-15 07:04:19,914::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:19,925::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:19,926::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-821::DEBUG::2014-10-15 07:04:19,927::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-821::DEBUG::2014-10-15 07:04:19,927::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:20,889::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:21,891::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:22,892::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:23,039::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:23,040::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-822::DEBUG::2014-10-15 07:04:23,041::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:23,042::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:23,043::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-823::DEBUG::2014-10-15 07:04:23,044::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-823::DEBUG::2014-10-15 07:04:23,044::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:23,893::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:23,905::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:23,905::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:23,905::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:04:24,895::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:25,896::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:26,103::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:26,104::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-824::DEBUG::2014-10-15 07:04:26,105::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:26,106::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:26,106::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-825::DEBUG::2014-10-15 07:04:26,107::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-825::DEBUG::2014-10-15 07:04:26,107::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:26,897::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-25::DEBUG::2014-10-15 07:04:27,180::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 07:04:27,180::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:27,181::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:27,181::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:27,181::__init__::232::IOProcess::(_processLogs) (678) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:04:27,181::__init__::232::IOProcess::(_processLogs) (678) Queuing response >Thread-25::DEBUG::2014-10-15 07:04:27,189::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 07:04:27,189::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:27,189::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:27,189::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:27,190::__init__::232::IOProcess::(_processLogs) (679) Got request for method 'touch' >Thread-16::DEBUG::2014-10-15 07:04:27,212::__init__::232::IOProcess::(_processLogs) (679) Queuing response >Thread-25::DEBUG::2014-10-15 07:04:27,213::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-16::DEBUG::2014-10-15 07:04:27,213::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:27,213::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:27,213::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:27,213::__init__::232::IOProcess::(_processLogs) (680) Got request for method 'readfile' >Thread-16::DEBUG::2014-10-15 07:04:27,214::__init__::232::IOProcess::(_processLogs) (680) Queuing response >Thread-25::DEBUG::2014-10-15 07:04:27,214::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=domain1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-0000000003e1', 'REMOTE_PATH=10.66.128.104:/home/NFS1', 'ROLE=Master', 'SDUUID=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b90f33781d11f6e6b0b3945de3167c0719a458c1'] >Thread-16::DEBUG::2014-10-15 07:04:27,214::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:27,215::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:27,215::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:27,215::__init__::232::IOProcess::(_processLogs) (681) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:27,215::__init__::232::IOProcess::(_processLogs) (681) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:27,215::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:27,215::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:27,215::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:27,216::__init__::232::IOProcess::(_processLogs) (682) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:04:27,216::__init__::232::IOProcess::(_processLogs) (682) Queuing response >Thread-25::DEBUG::2014-10-15 07:04:27,216::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-25::INFO::2014-10-15 07:04:27,216::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_imageNS already registered >Thread-25::INFO::2014-10-15 07:04:27,216::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_volumeNS already registered >Thread-16::DEBUG::2014-10-15 07:04:27,216::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:27,216::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:27,216::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:27,217::__init__::232::IOProcess::(_processLogs) (683) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:04:27,217::__init__::232::IOProcess::(_processLogs) (683) Queuing response >Thread-25::DEBUG::2014-10-15 07:04:27,217::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:04:27,223::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000358431 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:04:27,224::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:27,224::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:27,224::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:27,224::__init__::232::IOProcess::(_processLogs) (684) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:04:27,224::__init__::232::IOProcess::(_processLogs) (684) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:27,225::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:27,225::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:27,225::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:27,225::__init__::232::IOProcess::(_processLogs) (685) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:27,225::__init__::232::IOProcess::(_processLogs) (685) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:27,225::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:27,226::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:27,226::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:27,226::__init__::232::IOProcess::(_processLogs) (686) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:27,226::__init__::232::IOProcess::(_processLogs) (686) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:27,226::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:27,226::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:27,226::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:27,226::__init__::232::IOProcess::(_processLogs) (687) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:27,227::__init__::232::IOProcess::(_processLogs) (687) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:04:27,898::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:28,899::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:28,908::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:28,909::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:28,909::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:29,126::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:29,126::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-827::DEBUG::2014-10-15 07:04:29,127::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:29,129::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:29,129::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-828::DEBUG::2014-10-15 07:04:29,130::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-828::DEBUG::2014-10-15 07:04:29,130::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:29,901::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:30,902::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:31,903::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:32,149::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:32,151::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-829::DEBUG::2014-10-15 07:04:32,153::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:32,155::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:32,156::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-830::DEBUG::2014-10-15 07:04:32,158::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-830::DEBUG::2014-10-15 07:04:32,159::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:32,905::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:33,906::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:33,915::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:33,915::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:33,916::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:04:34,907::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:35,174::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:35,175::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-831::DEBUG::2014-10-15 07:04:35,176::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a29fa71-9295-48c9-8aaa-77a411881271`::moving from state init -> state preparing >Thread-831::INFO::2014-10-15 07:04:35,176::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-831::INFO::2014-10-15 07:04:35,176::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000358431', 'lastCheck': '7.9', 'valid': True}} >Thread-831::DEBUG::2014-10-15 07:04:35,177::task::1191::Storage.TaskManager.Task::(prepare) Task=`1a29fa71-9295-48c9-8aaa-77a411881271`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000358431', 'lastCheck': '7.9', 'valid': True}} >Thread-831::DEBUG::2014-10-15 07:04:35,177::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a29fa71-9295-48c9-8aaa-77a411881271`::moving from state preparing -> state finished >Thread-831::DEBUG::2014-10-15 07:04:35,177::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-831::DEBUG::2014-10-15 07:04:35,177::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-831::DEBUG::2014-10-15 07:04:35,177::task::993::Storage.TaskManager.Task::(_decref) Task=`1a29fa71-9295-48c9-8aaa-77a411881271`::ref 0 aborting False >Thread-831::ERROR::2014-10-15 07:04:35,178::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-831::DEBUG::2014-10-15 07:04:35,178::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:35,188::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:35,189::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-832::DEBUG::2014-10-15 07:04:35,189::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-832::DEBUG::2014-10-15 07:04:35,190::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:35,909::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:36,910::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:04:37,228::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:37,228::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:37,228::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:37,228::__init__::232::IOProcess::(_processLogs) (688) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:04:37,228::__init__::232::IOProcess::(_processLogs) (688) Queuing response >Thread-25::DEBUG::2014-10-15 07:04:37,229::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:04:37,235::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00035862 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:04:37,235::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:37,235::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:37,236::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:37,236::__init__::232::IOProcess::(_processLogs) (689) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:04:37,236::__init__::232::IOProcess::(_processLogs) (689) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:37,236::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:37,237::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:37,237::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:37,237::__init__::232::IOProcess::(_processLogs) (690) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:37,237::__init__::232::IOProcess::(_processLogs) (690) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:37,237::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:37,237::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:37,237::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:37,237::__init__::232::IOProcess::(_processLogs) (691) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:37,238::__init__::232::IOProcess::(_processLogs) (691) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:37,238::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:37,238::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:37,238::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:37,238::__init__::232::IOProcess::(_processLogs) (692) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:37,238::__init__::232::IOProcess::(_processLogs) (692) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:04:37,911::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:38,319::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:38,320::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-833::DEBUG::2014-10-15 07:04:38,321::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:38,322::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:38,322::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-834::DEBUG::2014-10-15 07:04:38,323::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-834::DEBUG::2014-10-15 07:04:38,324::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:38,912::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:38,912::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:04:38,913::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:38,917::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:38,917::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:38,918::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:04:39,914::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:40,915::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:41,351::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:41,352::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-835::DEBUG::2014-10-15 07:04:41,353::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:41,354::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:41,355::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-836::DEBUG::2014-10-15 07:04:41,356::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-836::DEBUG::2014-10-15 07:04:41,356::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:41,917::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:42,918::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:43,919::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:43,924::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:43,924::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:43,925::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:44,380::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:44,380::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-837::DEBUG::2014-10-15 07:04:44,382::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:44,383::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:44,383::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-838::DEBUG::2014-10-15 07:04:44,384::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-838::DEBUG::2014-10-15 07:04:44,384::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:44,920::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:45,922::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:46,923::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:04:47,239::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:47,239::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:47,240::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:47,240::__init__::232::IOProcess::(_processLogs) (693) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:04:47,240::__init__::232::IOProcess::(_processLogs) (693) Queuing response >Thread-25::DEBUG::2014-10-15 07:04:47,240::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:04:47,246::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000423892 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:04:47,247::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:47,247::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:47,247::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:47,247::__init__::232::IOProcess::(_processLogs) (694) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:04:47,248::__init__::232::IOProcess::(_processLogs) (694) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:47,248::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:47,248::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:47,248::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:47,248::__init__::232::IOProcess::(_processLogs) (695) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:47,249::__init__::232::IOProcess::(_processLogs) (695) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:47,249::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:47,249::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:47,249::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:47,249::__init__::232::IOProcess::(_processLogs) (696) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:47,249::__init__::232::IOProcess::(_processLogs) (696) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:47,250::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:47,250::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:47,250::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:47,250::__init__::232::IOProcess::(_processLogs) (697) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:47,250::__init__::232::IOProcess::(_processLogs) (697) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:47,405::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:47,405::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-839::DEBUG::2014-10-15 07:04:47,406::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:47,408::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:47,408::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-840::DEBUG::2014-10-15 07:04:47,409::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-840::DEBUG::2014-10-15 07:04:47,409::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:47,924::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:48,925::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:48,930::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:48,931::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:48,931::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:04:49,927::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:50,433::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:50,434::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-841::DEBUG::2014-10-15 07:04:50,435::task::595::Storage.TaskManager.Task::(_updateState) Task=`c76ed763-3e23-4023-9cd9-48fb06a949c0`::moving from state init -> state preparing >Thread-841::INFO::2014-10-15 07:04:50,436::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-841::INFO::2014-10-15 07:04:50,436::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423892', 'lastCheck': '3.2', 'valid': True}} >Thread-841::DEBUG::2014-10-15 07:04:50,436::task::1191::Storage.TaskManager.Task::(prepare) Task=`c76ed763-3e23-4023-9cd9-48fb06a949c0`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423892', 'lastCheck': '3.2', 'valid': True}} >Thread-841::DEBUG::2014-10-15 07:04:50,436::task::595::Storage.TaskManager.Task::(_updateState) Task=`c76ed763-3e23-4023-9cd9-48fb06a949c0`::moving from state preparing -> state finished >Thread-841::DEBUG::2014-10-15 07:04:50,436::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-841::DEBUG::2014-10-15 07:04:50,436::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-841::DEBUG::2014-10-15 07:04:50,436::task::993::Storage.TaskManager.Task::(_decref) Task=`c76ed763-3e23-4023-9cd9-48fb06a949c0`::ref 0 aborting False >Thread-841::ERROR::2014-10-15 07:04:50,437::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-841::DEBUG::2014-10-15 07:04:50,438::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:50,450::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:50,451::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-842::DEBUG::2014-10-15 07:04:50,452::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-842::DEBUG::2014-10-15 07:04:50,452::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:50,928::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:51,929::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:52,931::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:53,608::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:53,609::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-843::DEBUG::2014-10-15 07:04:53,610::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:53,611::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:53,611::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-844::DEBUG::2014-10-15 07:04:53,612::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-844::DEBUG::2014-10-15 07:04:53,613::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:53,932::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:53,937::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:53,938::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:53,938::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:04:54,933::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:55,934::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:56,646::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:56,647::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-845::DEBUG::2014-10-15 07:04:56,648::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:56,649::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:56,649::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-846::DEBUG::2014-10-15 07:04:56,650::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-846::DEBUG::2014-10-15 07:04:56,651::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:56,936::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:04:57,251::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:57,251::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:57,251::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:57,251::__init__::232::IOProcess::(_processLogs) (698) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:04:57,252::__init__::232::IOProcess::(_processLogs) (698) Queuing response >Thread-25::DEBUG::2014-10-15 07:04:57,252::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:04:57,258::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000383889 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:04:57,259::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:57,259::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:57,259::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:57,259::__init__::232::IOProcess::(_processLogs) (699) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:04:57,259::__init__::232::IOProcess::(_processLogs) (699) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:57,260::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:57,260::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:57,260::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:57,260::__init__::232::IOProcess::(_processLogs) (700) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:57,260::__init__::232::IOProcess::(_processLogs) (700) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:57,260::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:57,261::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:57,261::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:57,261::__init__::232::IOProcess::(_processLogs) (701) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:57,261::__init__::232::IOProcess::(_processLogs) (701) Queuing response >Thread-16::DEBUG::2014-10-15 07:04:57,261::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:04:57,261::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:04:57,261::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:04:57,261::__init__::232::IOProcess::(_processLogs) (702) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:04:57,261::__init__::232::IOProcess::(_processLogs) (702) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:04:57,937::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:04:58,938::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:58,944::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:58,944::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:04:58,945::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:59,669::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:59,670::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-847::DEBUG::2014-10-15 07:04:59,671::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:04:59,672::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:04:59,673::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-848::DEBUG::2014-10-15 07:04:59,673::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-848::DEBUG::2014-10-15 07:04:59,674::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:04:59,940::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:00,941::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:01,942::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:02,689::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:02,690::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-849::DEBUG::2014-10-15 07:05:02,690::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:02,691::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:02,692::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-850::DEBUG::2014-10-15 07:05:02,693::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-850::DEBUG::2014-10-15 07:05:02,693::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:02,943::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-17::DEBUG::2014-10-15 07:05:03,861::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e2a1875-d0fa-4211-9933-d29c962aa317`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 07:05:03,862::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:05:03,862::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:03,862::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:03,862::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:03,863::__init__::232::IOProcess::(_processLogs) (703) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:05:03,864::__init__::232::IOProcess::(_processLogs) (703) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:03,864::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:03,864::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:03,864::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:03,865::__init__::232::IOProcess::(_processLogs) (704) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:05:03,865::__init__::232::IOProcess::(_processLogs) (704) Queuing response >Thread-17::INFO::2014-10-15 07:05:03,865::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:05:03,865::task::1191::Storage.TaskManager.Task::(prepare) Task=`9e2a1875-d0fa-4211-9933-d29c962aa317`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:05:03,865::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e2a1875-d0fa-4211-9933-d29c962aa317`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 07:05:03,865::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 07:05:03,865::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 07:05:03,865::task::993::Storage.TaskManager.Task::(_decref) Task=`9e2a1875-d0fa-4211-9933-d29c962aa317`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 07:05:03,944::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:03,951::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:03,951::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:03,952::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:05:04,946::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:05,720::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:05,720::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-851::DEBUG::2014-10-15 07:05:05,722::task::595::Storage.TaskManager.Task::(_updateState) Task=`79a1103e-d2d0-473d-936f-8ce9db52073f`::moving from state init -> state preparing >Thread-851::INFO::2014-10-15 07:05:05,722::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-851::INFO::2014-10-15 07:05:05,722::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000383889', 'lastCheck': '8.5', 'valid': True}} >Thread-851::DEBUG::2014-10-15 07:05:05,722::task::1191::Storage.TaskManager.Task::(prepare) Task=`79a1103e-d2d0-473d-936f-8ce9db52073f`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000383889', 'lastCheck': '8.5', 'valid': True}} >Thread-851::DEBUG::2014-10-15 07:05:05,722::task::595::Storage.TaskManager.Task::(_updateState) Task=`79a1103e-d2d0-473d-936f-8ce9db52073f`::moving from state preparing -> state finished >Thread-851::DEBUG::2014-10-15 07:05:05,722::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-851::DEBUG::2014-10-15 07:05:05,722::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-851::DEBUG::2014-10-15 07:05:05,723::task::993::Storage.TaskManager.Task::(_decref) Task=`79a1103e-d2d0-473d-936f-8ce9db52073f`::ref 0 aborting False >Thread-851::ERROR::2014-10-15 07:05:05,723::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-851::DEBUG::2014-10-15 07:05:05,724::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:05,737::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:05,738::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-852::DEBUG::2014-10-15 07:05:05,739::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-852::DEBUG::2014-10-15 07:05:05,739::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:05,947::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:06,948::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:05:07,262::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:07,263::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:07,263::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:07,263::__init__::232::IOProcess::(_processLogs) (705) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:05:07,263::__init__::232::IOProcess::(_processLogs) (705) Queuing response >Thread-25::DEBUG::2014-10-15 07:05:07,264::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:05:07,270::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000442048 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:05:07,271::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:07,271::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:07,271::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:07,271::__init__::232::IOProcess::(_processLogs) (706) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:05:07,271::__init__::232::IOProcess::(_processLogs) (706) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:07,272::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:07,272::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:07,272::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:07,272::__init__::232::IOProcess::(_processLogs) (707) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:07,272::__init__::232::IOProcess::(_processLogs) (707) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:07,273::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:07,273::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:07,273::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:07,273::__init__::232::IOProcess::(_processLogs) (708) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:07,273::__init__::232::IOProcess::(_processLogs) (708) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:07,274::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:07,274::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:07,274::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:07,274::__init__::232::IOProcess::(_processLogs) (709) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:07,274::__init__::232::IOProcess::(_processLogs) (709) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:05:07,949::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:08,856::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:08,857::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-853::DEBUG::2014-10-15 07:05:08,858::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:08,859::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:08,860::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-854::DEBUG::2014-10-15 07:05:08,861::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-854::DEBUG::2014-10-15 07:05:08,861::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:08,952::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:08,953::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:05:08,953::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:08,957::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:08,958::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:08,958::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:05:09,954::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:10,956::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:11,892::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:11,893::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-855::DEBUG::2014-10-15 07:05:11,894::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:11,898::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:11,899::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-856::DEBUG::2014-10-15 07:05:11,900::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-856::DEBUG::2014-10-15 07:05:11,900::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:11,957::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:12,958::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:13,959::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:13,960::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:13,961::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:13,961::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:14,927::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:14,927::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-857::DEBUG::2014-10-15 07:05:14,929::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:14,930::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:14,930::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-858::DEBUG::2014-10-15 07:05:14,931::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-858::DEBUG::2014-10-15 07:05:14,931::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:14,961::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:15,963::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:16,964::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:05:17,275::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:17,275::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:17,275::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:17,275::__init__::232::IOProcess::(_processLogs) (710) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:05:17,275::__init__::232::IOProcess::(_processLogs) (710) Queuing response >Thread-25::DEBUG::2014-10-15 07:05:17,276::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:05:17,282::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000373176 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:05:17,282::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:17,283::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:17,283::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:17,283::__init__::232::IOProcess::(_processLogs) (711) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:05:17,283::__init__::232::IOProcess::(_processLogs) (711) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:17,284::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:17,284::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:17,284::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:17,284::__init__::232::IOProcess::(_processLogs) (712) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:17,284::__init__::232::IOProcess::(_processLogs) (712) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:17,284::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:17,284::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:17,284::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:17,285::__init__::232::IOProcess::(_processLogs) (713) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:17,285::__init__::232::IOProcess::(_processLogs) (713) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:17,285::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:17,285::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:17,285::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:17,285::__init__::232::IOProcess::(_processLogs) (714) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:17,285::__init__::232::IOProcess::(_processLogs) (714) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:17,950::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:17,951::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-859::DEBUG::2014-10-15 07:05:17,952::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:17,953::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:17,954::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-860::DEBUG::2014-10-15 07:05:17,955::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-860::DEBUG::2014-10-15 07:05:17,955::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:17,965::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:18,964::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:18,965::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:18,965::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:05:18,966::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:19,967::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:20,968::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:20,970::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:20,971::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-861::DEBUG::2014-10-15 07:05:20,972::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f4130ba-7344-440f-bb69-513393501774`::moving from state init -> state preparing >Thread-861::INFO::2014-10-15 07:05:20,972::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-861::INFO::2014-10-15 07:05:20,972::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000373176', 'lastCheck': '3.7', 'valid': True}} >Thread-861::DEBUG::2014-10-15 07:05:20,972::task::1191::Storage.TaskManager.Task::(prepare) Task=`0f4130ba-7344-440f-bb69-513393501774`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000373176', 'lastCheck': '3.7', 'valid': True}} >Thread-861::DEBUG::2014-10-15 07:05:20,972::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f4130ba-7344-440f-bb69-513393501774`::moving from state preparing -> state finished >Thread-861::DEBUG::2014-10-15 07:05:20,973::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-861::DEBUG::2014-10-15 07:05:20,973::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-861::DEBUG::2014-10-15 07:05:20,973::task::993::Storage.TaskManager.Task::(_decref) Task=`0f4130ba-7344-440f-bb69-513393501774`::ref 0 aborting False >Thread-861::ERROR::2014-10-15 07:05:20,974::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-861::DEBUG::2014-10-15 07:05:20,974::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:20,984::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:20,984::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-862::DEBUG::2014-10-15 07:05:20,986::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-862::DEBUG::2014-10-15 07:05:20,986::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:21,969::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:22,970::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:23,971::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:23,972::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:23,973::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:23,973::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:24,084::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:24,085::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-863::DEBUG::2014-10-15 07:05:24,086::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:24,087::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:24,088::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-864::DEBUG::2014-10-15 07:05:24,089::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-864::DEBUG::2014-10-15 07:05:24,089::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:24,973::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:25,975::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:26,976::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:27,126::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:27,127::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-865::DEBUG::2014-10-15 07:05:27,128::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:27,129::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:27,130::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-866::DEBUG::2014-10-15 07:05:27,131::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-866::DEBUG::2014-10-15 07:05:27,131::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:05:27,286::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:27,286::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:27,286::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:27,287::__init__::232::IOProcess::(_processLogs) (715) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:05:27,287::__init__::232::IOProcess::(_processLogs) (715) Queuing response >Thread-25::DEBUG::2014-10-15 07:05:27,287::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:05:27,293::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000386512 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:05:27,294::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:27,294::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:27,294::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:27,294::__init__::232::IOProcess::(_processLogs) (716) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:05:27,294::__init__::232::IOProcess::(_processLogs) (716) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:27,295::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:27,295::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:27,295::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:27,295::__init__::232::IOProcess::(_processLogs) (717) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:27,295::__init__::232::IOProcess::(_processLogs) (717) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:27,296::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:27,296::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:27,296::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:27,296::__init__::232::IOProcess::(_processLogs) (718) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:27,296::__init__::232::IOProcess::(_processLogs) (718) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:27,296::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:27,296::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:27,296::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:27,297::__init__::232::IOProcess::(_processLogs) (719) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:27,297::__init__::232::IOProcess::(_processLogs) (719) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:05:27,977::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:28,977::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:28,977::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:05:28,978::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:28,978::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:05:29,980::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:30,155::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:30,155::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-867::DEBUG::2014-10-15 07:05:30,156::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:30,158::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:30,158::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-868::DEBUG::2014-10-15 07:05:30,159::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-868::DEBUG::2014-10-15 07:05:30,159::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:30,981::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:31,981::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:32,983::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:33,195::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:33,196::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-869::DEBUG::2014-10-15 07:05:33,197::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:33,198::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:33,199::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-870::DEBUG::2014-10-15 07:05:33,200::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-870::DEBUG::2014-10-15 07:05:33,200::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:33,984::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:33,985::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:33,985::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:33,986::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:05:34,986::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:35,987::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:36,221::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:36,222::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-871::DEBUG::2014-10-15 07:05:36,223::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e4a2adc-55cc-4f7d-bcc7-861cddce09f8`::moving from state init -> state preparing >Thread-871::INFO::2014-10-15 07:05:36,223::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-871::INFO::2014-10-15 07:05:36,223::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000386512', 'lastCheck': '8.9', 'valid': True}} >Thread-871::DEBUG::2014-10-15 07:05:36,223::task::1191::Storage.TaskManager.Task::(prepare) Task=`9e4a2adc-55cc-4f7d-bcc7-861cddce09f8`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000386512', 'lastCheck': '8.9', 'valid': True}} >Thread-871::DEBUG::2014-10-15 07:05:36,223::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e4a2adc-55cc-4f7d-bcc7-861cddce09f8`::moving from state preparing -> state finished >Thread-871::DEBUG::2014-10-15 07:05:36,223::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-871::DEBUG::2014-10-15 07:05:36,223::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-871::DEBUG::2014-10-15 07:05:36,224::task::993::Storage.TaskManager.Task::(_decref) Task=`9e4a2adc-55cc-4f7d-bcc7-861cddce09f8`::ref 0 aborting False >Thread-871::ERROR::2014-10-15 07:05:36,224::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-871::DEBUG::2014-10-15 07:05:36,225::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:36,238::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:36,238::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-872::DEBUG::2014-10-15 07:05:36,239::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-872::DEBUG::2014-10-15 07:05:36,240::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:36,989::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:05:37,298::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:37,298::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:37,298::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:37,298::__init__::232::IOProcess::(_processLogs) (720) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:05:37,298::__init__::232::IOProcess::(_processLogs) (720) Queuing response >Thread-25::DEBUG::2014-10-15 07:05:37,299::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:05:37,305::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00040406 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:05:37,305::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:37,306::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:37,306::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:37,306::__init__::232::IOProcess::(_processLogs) (721) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:05:37,306::__init__::232::IOProcess::(_processLogs) (721) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:37,307::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:37,307::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:37,307::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:37,307::__init__::232::IOProcess::(_processLogs) (722) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:37,307::__init__::232::IOProcess::(_processLogs) (722) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:37,307::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:37,307::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:37,307::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:37,308::__init__::232::IOProcess::(_processLogs) (723) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:37,308::__init__::232::IOProcess::(_processLogs) (723) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:37,308::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:37,308::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:37,308::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:37,308::__init__::232::IOProcess::(_processLogs) (724) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:37,308::__init__::232::IOProcess::(_processLogs) (724) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:05:37,990::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:38,991::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:38,992::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:38,992::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:05:38,993::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:38,994::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:38,994::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:39,359::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:39,360::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-873::DEBUG::2014-10-15 07:05:39,361::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:39,362::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:39,363::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-874::DEBUG::2014-10-15 07:05:39,364::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-874::DEBUG::2014-10-15 07:05:39,364::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:39,995::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:40,996::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:41,997::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:42,409::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:42,410::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-875::DEBUG::2014-10-15 07:05:42,411::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:42,412::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:42,412::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-876::DEBUG::2014-10-15 07:05:42,413::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-876::DEBUG::2014-10-15 07:05:42,414::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:42,999::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:44,000::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:44,001::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:44,001::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:44,002::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:05:45,002::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:45,434::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:45,435::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-877::DEBUG::2014-10-15 07:05:45,436::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:45,437::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:45,438::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-878::DEBUG::2014-10-15 07:05:45,439::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-878::DEBUG::2014-10-15 07:05:45,439::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:46,003::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:47,004::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:05:47,309::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:47,310::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:47,310::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:47,310::__init__::232::IOProcess::(_processLogs) (725) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:05:47,310::__init__::232::IOProcess::(_processLogs) (725) Queuing response >Thread-25::DEBUG::2014-10-15 07:05:47,310::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:05:47,316::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000444739 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:05:47,317::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:47,317::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:47,317::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:47,317::__init__::232::IOProcess::(_processLogs) (726) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:05:47,318::__init__::232::IOProcess::(_processLogs) (726) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:47,318::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:47,318::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:47,318::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:47,318::__init__::232::IOProcess::(_processLogs) (727) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:47,318::__init__::232::IOProcess::(_processLogs) (727) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:47,319::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:47,319::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:47,319::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:47,320::__init__::232::IOProcess::(_processLogs) (728) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:47,320::__init__::232::IOProcess::(_processLogs) (728) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:47,321::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:47,321::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:47,321::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:47,321::__init__::232::IOProcess::(_processLogs) (729) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:47,321::__init__::232::IOProcess::(_processLogs) (729) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:05:48,006::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:48,456::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:48,457::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-879::DEBUG::2014-10-15 07:05:48,458::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:48,459::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:48,460::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-880::DEBUG::2014-10-15 07:05:48,461::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-880::DEBUG::2014-10-15 07:05:48,461::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:49,007::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:49,007::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:49,008::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:49,008::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:05:50,008::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:51,010::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:51,479::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:51,480::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-881::DEBUG::2014-10-15 07:05:51,481::task::595::Storage.TaskManager.Task::(_updateState) Task=`13fb75ab-5983-4e65-91ec-0be5dc37ad90`::moving from state init -> state preparing >Thread-881::INFO::2014-10-15 07:05:51,481::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-881::INFO::2014-10-15 07:05:51,482::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000444739', 'lastCheck': '4.2', 'valid': True}} >Thread-881::DEBUG::2014-10-15 07:05:51,482::task::1191::Storage.TaskManager.Task::(prepare) Task=`13fb75ab-5983-4e65-91ec-0be5dc37ad90`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000444739', 'lastCheck': '4.2', 'valid': True}} >Thread-881::DEBUG::2014-10-15 07:05:51,482::task::595::Storage.TaskManager.Task::(_updateState) Task=`13fb75ab-5983-4e65-91ec-0be5dc37ad90`::moving from state preparing -> state finished >Thread-881::DEBUG::2014-10-15 07:05:51,482::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-881::DEBUG::2014-10-15 07:05:51,482::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-881::DEBUG::2014-10-15 07:05:51,482::task::993::Storage.TaskManager.Task::(_decref) Task=`13fb75ab-5983-4e65-91ec-0be5dc37ad90`::ref 0 aborting False >Thread-881::ERROR::2014-10-15 07:05:51,483::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-881::DEBUG::2014-10-15 07:05:51,484::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:51,494::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:51,494::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-882::DEBUG::2014-10-15 07:05:51,496::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-882::DEBUG::2014-10-15 07:05:51,496::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:52,011::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:53,012::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:54,013::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:54,014::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:54,015::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:54,015::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:54,720::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:54,721::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-883::DEBUG::2014-10-15 07:05:54,722::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:54,723::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:54,723::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-884::DEBUG::2014-10-15 07:05:54,724::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-884::DEBUG::2014-10-15 07:05:54,725::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:55,015::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:56,016::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:57,017::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:05:57,322::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:57,322::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:57,322::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:57,323::__init__::232::IOProcess::(_processLogs) (730) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:05:57,323::__init__::232::IOProcess::(_processLogs) (730) Queuing response >Thread-25::DEBUG::2014-10-15 07:05:57,323::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:05:57,329::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000430514 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:05:57,330::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:57,330::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:57,330::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:57,330::__init__::232::IOProcess::(_processLogs) (731) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:05:57,330::__init__::232::IOProcess::(_processLogs) (731) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:57,331::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:57,331::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:57,331::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:57,331::__init__::232::IOProcess::(_processLogs) (732) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:57,331::__init__::232::IOProcess::(_processLogs) (732) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:57,332::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:57,332::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:57,332::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:57,332::__init__::232::IOProcess::(_processLogs) (733) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:57,332::__init__::232::IOProcess::(_processLogs) (733) Queuing response >Thread-16::DEBUG::2014-10-15 07:05:57,332::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:05:57,333::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:05:57,333::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:05:57,333::__init__::232::IOProcess::(_processLogs) (734) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:05:57,333::__init__::232::IOProcess::(_processLogs) (734) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:57,753::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:57,753::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-885::DEBUG::2014-10-15 07:05:57,755::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:05:57,756::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:05:57,756::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-886::DEBUG::2014-10-15 07:05:57,757::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-886::DEBUG::2014-10-15 07:05:57,757::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:05:58,018::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:05:59,020::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:59,021::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:59,021::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:05:59,021::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:06:00,022::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:00,773::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:00,774::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-887::DEBUG::2014-10-15 07:06:00,775::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:00,776::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:00,777::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-888::DEBUG::2014-10-15 07:06:00,778::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-888::DEBUG::2014-10-15 07:06:00,778::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:01,023::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:02,024::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:03,026::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:03,801::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:03,802::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-889::DEBUG::2014-10-15 07:06:03,803::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:03,804::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:03,805::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-890::DEBUG::2014-10-15 07:06:03,805::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-890::DEBUG::2014-10-15 07:06:03,806::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-17::DEBUG::2014-10-15 07:06:03,904::task::595::Storage.TaskManager.Task::(_updateState) Task=`19c698a9-212a-4c6f-9bde-232814a754e8`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 07:06:03,905::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:06:03,905::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:03,905::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:03,905::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:03,906::__init__::232::IOProcess::(_processLogs) (735) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:06:03,906::__init__::232::IOProcess::(_processLogs) (735) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:03,907::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:03,907::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:03,907::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:03,907::__init__::232::IOProcess::(_processLogs) (736) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:06:03,907::__init__::232::IOProcess::(_processLogs) (736) Queuing response >Thread-17::INFO::2014-10-15 07:06:03,908::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:06:03,908::task::1191::Storage.TaskManager.Task::(prepare) Task=`19c698a9-212a-4c6f-9bde-232814a754e8`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:06:03,908::task::595::Storage.TaskManager.Task::(_updateState) Task=`19c698a9-212a-4c6f-9bde-232814a754e8`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 07:06:03,908::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 07:06:03,908::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 07:06:03,908::task::993::Storage.TaskManager.Task::(_decref) Task=`19c698a9-212a-4c6f-9bde-232814a754e8`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:04,027::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:06:04,027::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:04,028::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:04,029::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:06:05,029::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:06,030::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:06,832::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:06,833::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-891::DEBUG::2014-10-15 07:06:06,834::task::595::Storage.TaskManager.Task::(_updateState) Task=`be2d42e2-8c85-4b48-90bb-0012f5c772b4`::moving from state init -> state preparing >Thread-891::INFO::2014-10-15 07:06:06,834::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-891::INFO::2014-10-15 07:06:06,834::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000430514', 'lastCheck': '9.5', 'valid': True}} >Thread-891::DEBUG::2014-10-15 07:06:06,834::task::1191::Storage.TaskManager.Task::(prepare) Task=`be2d42e2-8c85-4b48-90bb-0012f5c772b4`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000430514', 'lastCheck': '9.5', 'valid': True}} >Thread-891::DEBUG::2014-10-15 07:06:06,834::task::595::Storage.TaskManager.Task::(_updateState) Task=`be2d42e2-8c85-4b48-90bb-0012f5c772b4`::moving from state preparing -> state finished >Thread-891::DEBUG::2014-10-15 07:06:06,835::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-891::DEBUG::2014-10-15 07:06:06,835::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-891::DEBUG::2014-10-15 07:06:06,835::task::993::Storage.TaskManager.Task::(_decref) Task=`be2d42e2-8c85-4b48-90bb-0012f5c772b4`::ref 0 aborting False >Thread-891::ERROR::2014-10-15 07:06:06,836::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-891::DEBUG::2014-10-15 07:06:06,836::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:06,844::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:06,845::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-892::DEBUG::2014-10-15 07:06:06,846::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-892::DEBUG::2014-10-15 07:06:06,846::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:07,032::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:06:07,334::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:07,334::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:07,334::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:07,334::__init__::232::IOProcess::(_processLogs) (737) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:06:07,334::__init__::232::IOProcess::(_processLogs) (737) Queuing response >Thread-25::DEBUG::2014-10-15 07:06:07,334::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:06:07,341::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000381038 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:06:07,341::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:07,342::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:07,342::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:07,342::__init__::232::IOProcess::(_processLogs) (738) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:06:07,342::__init__::232::IOProcess::(_processLogs) (738) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:07,342::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:07,343::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:07,343::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:07,343::__init__::232::IOProcess::(_processLogs) (739) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:07,343::__init__::232::IOProcess::(_processLogs) (739) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:07,343::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:07,343::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:07,343::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:07,343::__init__::232::IOProcess::(_processLogs) (740) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:07,344::__init__::232::IOProcess::(_processLogs) (740) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:07,344::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:07,344::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:07,344::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:07,344::__init__::232::IOProcess::(_processLogs) (741) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:07,344::__init__::232::IOProcess::(_processLogs) (741) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:06:08,033::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:09,034::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:09,035::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:06:09,036::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:06:09,036::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:09,037::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:09,038::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:09,985::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:09,986::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-893::DEBUG::2014-10-15 07:06:09,987::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:09,989::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:09,989::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-894::DEBUG::2014-10-15 07:06:09,990::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-894::DEBUG::2014-10-15 07:06:09,990::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:10,038::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:11,039::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:12,041::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:13,018::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:13,019::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-895::DEBUG::2014-10-15 07:06:13,020::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:13,021::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:13,022::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-896::DEBUG::2014-10-15 07:06:13,022::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-896::DEBUG::2014-10-15 07:06:13,023::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:13,042::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:14,043::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:14,044::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:14,045::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:14,045::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:06:15,045::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:16,043::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:16,044::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-897::DEBUG::2014-10-15 07:06:16,045::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:16,046::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >VM Channels Listener::DEBUG::2014-10-15 07:06:16,047::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpcServer::DEBUG::2014-10-15 07:06:16,047::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-898::DEBUG::2014-10-15 07:06:16,048::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-898::DEBUG::2014-10-15 07:06:16,049::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:17,049::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:06:17,345::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:17,345::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:17,345::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:17,346::__init__::232::IOProcess::(_processLogs) (742) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:06:17,346::__init__::232::IOProcess::(_processLogs) (742) Queuing response >Thread-25::DEBUG::2014-10-15 07:06:17,346::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:06:17,352::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000425598 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:06:17,353::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:17,353::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:17,353::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:17,353::__init__::232::IOProcess::(_processLogs) (743) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:06:17,353::__init__::232::IOProcess::(_processLogs) (743) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:17,354::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:17,354::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:17,354::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:17,354::__init__::232::IOProcess::(_processLogs) (744) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:17,354::__init__::232::IOProcess::(_processLogs) (744) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:17,355::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:17,355::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:17,355::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:17,355::__init__::232::IOProcess::(_processLogs) (745) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:17,355::__init__::232::IOProcess::(_processLogs) (745) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:17,356::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:17,356::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:17,356::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:17,356::__init__::232::IOProcess::(_processLogs) (746) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:17,356::__init__::232::IOProcess::(_processLogs) (746) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:06:18,050::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:19,051::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:06:19,051::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:19,052::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:19,052::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:19,095::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:19,096::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-899::DEBUG::2014-10-15 07:06:19,097::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:19,098::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:19,098::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-900::DEBUG::2014-10-15 07:06:19,099::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-900::DEBUG::2014-10-15 07:06:19,099::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:20,053::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:21,055::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:22,056::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:22,114::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:22,115::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-901::DEBUG::2014-10-15 07:06:22,116::task::595::Storage.TaskManager.Task::(_updateState) Task=`849972c7-f3b6-4b6f-8c17-77936b8d5cd6`::moving from state init -> state preparing >Thread-901::INFO::2014-10-15 07:06:22,117::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-901::INFO::2014-10-15 07:06:22,117::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425598', 'lastCheck': '4.8', 'valid': True}} >Thread-901::DEBUG::2014-10-15 07:06:22,117::task::1191::Storage.TaskManager.Task::(prepare) Task=`849972c7-f3b6-4b6f-8c17-77936b8d5cd6`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425598', 'lastCheck': '4.8', 'valid': True}} >Thread-901::DEBUG::2014-10-15 07:06:22,117::task::595::Storage.TaskManager.Task::(_updateState) Task=`849972c7-f3b6-4b6f-8c17-77936b8d5cd6`::moving from state preparing -> state finished >Thread-901::DEBUG::2014-10-15 07:06:22,117::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-901::DEBUG::2014-10-15 07:06:22,117::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-901::DEBUG::2014-10-15 07:06:22,117::task::993::Storage.TaskManager.Task::(_decref) Task=`849972c7-f3b6-4b6f-8c17-77936b8d5cd6`::ref 0 aborting False >Thread-901::ERROR::2014-10-15 07:06:22,118::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-901::DEBUG::2014-10-15 07:06:22,119::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:22,132::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:22,132::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-902::DEBUG::2014-10-15 07:06:22,133::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-902::DEBUG::2014-10-15 07:06:22,134::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:23,057::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:24,058::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:24,059::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:24,059::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:24,060::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:06:25,060::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:25,267::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:25,268::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-903::DEBUG::2014-10-15 07:06:25,269::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:25,270::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:25,271::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-904::DEBUG::2014-10-15 07:06:25,272::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-904::DEBUG::2014-10-15 07:06:25,272::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:26,062::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:27,063::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:06:27,357::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:27,357::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:27,357::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:27,357::__init__::232::IOProcess::(_processLogs) (747) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:06:27,357::__init__::232::IOProcess::(_processLogs) (747) Queuing response >Thread-25::DEBUG::2014-10-15 07:06:27,358::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:06:27,364::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00036533 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:06:27,364::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:27,364::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:27,364::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:27,365::__init__::232::IOProcess::(_processLogs) (748) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:06:27,365::__init__::232::IOProcess::(_processLogs) (748) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:27,365::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:27,365::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:27,366::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:27,366::__init__::232::IOProcess::(_processLogs) (749) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:27,366::__init__::232::IOProcess::(_processLogs) (749) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:27,366::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:27,366::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:27,366::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:27,366::__init__::232::IOProcess::(_processLogs) (750) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:27,366::__init__::232::IOProcess::(_processLogs) (750) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:27,367::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:27,367::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:27,367::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:27,367::__init__::232::IOProcess::(_processLogs) (751) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:27,367::__init__::232::IOProcess::(_processLogs) (751) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:06:28,064::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:28,304::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:28,305::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-905::DEBUG::2014-10-15 07:06:28,306::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:28,307::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:28,307::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-906::DEBUG::2014-10-15 07:06:28,308::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-906::DEBUG::2014-10-15 07:06:28,309::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:29,062::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:29,062::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:29,063::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:06:29,064::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:30,066::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:31,067::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:31,328::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:31,329::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-907::DEBUG::2014-10-15 07:06:31,330::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:31,331::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:31,332::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-908::DEBUG::2014-10-15 07:06:31,333::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-908::DEBUG::2014-10-15 07:06:31,333::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:32,068::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:33,069::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:34,069::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:34,069::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:34,070::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:06:34,071::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:34,358::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:34,358::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-909::DEBUG::2014-10-15 07:06:34,359::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:34,360::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:34,361::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-910::DEBUG::2014-10-15 07:06:34,362::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-910::DEBUG::2014-10-15 07:06:34,362::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:35,072::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:36,073::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:37,074::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:06:37,368::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:37,368::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:37,368::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:37,369::__init__::232::IOProcess::(_processLogs) (752) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:06:37,369::__init__::232::IOProcess::(_processLogs) (752) Queuing response >Thread-25::DEBUG::2014-10-15 07:06:37,369::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:06:37,375::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000454818 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:06:37,376::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:37,376::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:37,376::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:37,376::__init__::232::IOProcess::(_processLogs) (753) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:06:37,376::__init__::232::IOProcess::(_processLogs) (753) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:37,377::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:37,377::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:37,377::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:37,377::__init__::232::IOProcess::(_processLogs) (754) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:37,377::__init__::232::IOProcess::(_processLogs) (754) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:37,378::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:37,378::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:37,378::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:37,378::__init__::232::IOProcess::(_processLogs) (755) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:37,378::__init__::232::IOProcess::(_processLogs) (755) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:37,378::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:37,379::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:37,379::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:37,379::__init__::232::IOProcess::(_processLogs) (756) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:37,379::__init__::232::IOProcess::(_processLogs) (756) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:37,380::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:37,380::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-911::DEBUG::2014-10-15 07:06:37,381::task::595::Storage.TaskManager.Task::(_updateState) Task=`f5d3f609-1e58-4aa1-be90-cdb4f9169d51`::moving from state init -> state preparing >Thread-911::INFO::2014-10-15 07:06:37,382::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-911::INFO::2014-10-15 07:06:37,382::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000454818', 'lastCheck': '0.0', 'valid': True}} >Thread-911::DEBUG::2014-10-15 07:06:37,382::task::1191::Storage.TaskManager.Task::(prepare) Task=`f5d3f609-1e58-4aa1-be90-cdb4f9169d51`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000454818', 'lastCheck': '0.0', 'valid': True}} >Thread-911::DEBUG::2014-10-15 07:06:37,382::task::595::Storage.TaskManager.Task::(_updateState) Task=`f5d3f609-1e58-4aa1-be90-cdb4f9169d51`::moving from state preparing -> state finished >Thread-911::DEBUG::2014-10-15 07:06:37,382::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-911::DEBUG::2014-10-15 07:06:37,382::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-911::DEBUG::2014-10-15 07:06:37,382::task::993::Storage.TaskManager.Task::(_decref) Task=`f5d3f609-1e58-4aa1-be90-cdb4f9169d51`::ref 0 aborting False >Thread-911::ERROR::2014-10-15 07:06:37,383::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-911::DEBUG::2014-10-15 07:06:37,384::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:37,395::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:37,396::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-912::DEBUG::2014-10-15 07:06:37,397::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-912::DEBUG::2014-10-15 07:06:37,397::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:38,076::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:39,072::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:39,073::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:39,073::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:06:39,077::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:39,077::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:06:39,077::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >VM Channels Listener::DEBUG::2014-10-15 07:06:40,078::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:40,544::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:40,544::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-913::DEBUG::2014-10-15 07:06:40,545::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:40,547::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:40,547::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-914::DEBUG::2014-10-15 07:06:40,548::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-914::DEBUG::2014-10-15 07:06:40,549::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:41,080::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:42,081::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:43,082::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:43,580::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:43,581::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-915::DEBUG::2014-10-15 07:06:43,582::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:43,583::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:43,584::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-916::DEBUG::2014-10-15 07:06:43,585::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-916::DEBUG::2014-10-15 07:06:43,585::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:44,079::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:44,079::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:44,080::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:06:44,084::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:45,085::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:46,086::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:46,611::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:46,612::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-917::DEBUG::2014-10-15 07:06:46,613::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:46,614::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:46,614::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-918::DEBUG::2014-10-15 07:06:46,615::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-918::DEBUG::2014-10-15 07:06:46,616::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:47,087::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:06:47,380::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:47,380::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:47,380::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:47,380::__init__::232::IOProcess::(_processLogs) (757) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:06:47,380::__init__::232::IOProcess::(_processLogs) (757) Queuing response >Thread-25::DEBUG::2014-10-15 07:06:47,381::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:06:47,387::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00039163 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:06:47,387::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:47,387::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:47,388::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:47,388::__init__::232::IOProcess::(_processLogs) (758) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:06:47,388::__init__::232::IOProcess::(_processLogs) (758) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:47,388::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:47,388::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:47,388::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:47,389::__init__::232::IOProcess::(_processLogs) (759) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:47,389::__init__::232::IOProcess::(_processLogs) (759) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:47,389::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:47,389::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:47,389::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:47,389::__init__::232::IOProcess::(_processLogs) (760) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:47,389::__init__::232::IOProcess::(_processLogs) (760) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:47,390::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:47,390::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:47,390::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:47,390::__init__::232::IOProcess::(_processLogs) (761) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:47,390::__init__::232::IOProcess::(_processLogs) (761) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:06:48,089::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:49,081::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:49,081::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:49,082::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:06:49,090::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:49,642::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:49,643::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-919::DEBUG::2014-10-15 07:06:49,644::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:49,645::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:49,646::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-920::DEBUG::2014-10-15 07:06:49,647::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-920::DEBUG::2014-10-15 07:06:49,647::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:50,091::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:51,092::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:52,094::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:52,695::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:52,696::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-921::DEBUG::2014-10-15 07:06:52,697::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c06863d-87a4-40f7-990b-fa56821857b7`::moving from state init -> state preparing >Thread-921::INFO::2014-10-15 07:06:52,697::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-921::INFO::2014-10-15 07:06:52,697::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00039163', 'lastCheck': '5.3', 'valid': True}} >Thread-921::DEBUG::2014-10-15 07:06:52,697::task::1191::Storage.TaskManager.Task::(prepare) Task=`2c06863d-87a4-40f7-990b-fa56821857b7`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00039163', 'lastCheck': '5.3', 'valid': True}} >Thread-921::DEBUG::2014-10-15 07:06:52,697::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c06863d-87a4-40f7-990b-fa56821857b7`::moving from state preparing -> state finished >Thread-921::DEBUG::2014-10-15 07:06:52,697::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-921::DEBUG::2014-10-15 07:06:52,697::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-921::DEBUG::2014-10-15 07:06:52,698::task::993::Storage.TaskManager.Task::(_decref) Task=`2c06863d-87a4-40f7-990b-fa56821857b7`::ref 0 aborting False >Thread-921::ERROR::2014-10-15 07:06:52,698::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-921::DEBUG::2014-10-15 07:06:52,699::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:52,709::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:52,709::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-922::DEBUG::2014-10-15 07:06:52,710::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-922::DEBUG::2014-10-15 07:06:52,710::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:53,095::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:54,088::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:54,088::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:54,089::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:06:54,096::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:55,097::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:55,848::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:55,849::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-923::DEBUG::2014-10-15 07:06:55,850::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:55,851::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:55,852::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-924::DEBUG::2014-10-15 07:06:55,852::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-924::DEBUG::2014-10-15 07:06:55,853::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:06:56,099::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:06:57,100::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:06:57,391::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:57,391::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:57,391::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:57,391::__init__::232::IOProcess::(_processLogs) (762) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:06:57,392::__init__::232::IOProcess::(_processLogs) (762) Queuing response >Thread-25::DEBUG::2014-10-15 07:06:57,392::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:06:57,398::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000443964 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:06:57,399::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:57,399::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:57,399::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:57,399::__init__::232::IOProcess::(_processLogs) (763) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:06:57,399::__init__::232::IOProcess::(_processLogs) (763) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:57,400::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:57,400::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:57,400::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:57,400::__init__::232::IOProcess::(_processLogs) (764) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:57,400::__init__::232::IOProcess::(_processLogs) (764) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:57,401::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:57,401::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:57,401::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:57,401::__init__::232::IOProcess::(_processLogs) (765) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:57,401::__init__::232::IOProcess::(_processLogs) (765) Queuing response >Thread-16::DEBUG::2014-10-15 07:06:57,402::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:06:57,402::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:06:57,402::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:06:57,402::__init__::232::IOProcess::(_processLogs) (766) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:06:57,402::__init__::232::IOProcess::(_processLogs) (766) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:06:58,101::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:58,893::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:58,894::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-925::DEBUG::2014-10-15 07:06:58,895::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:06:58,896::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:06:58,897::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-926::DEBUG::2014-10-15 07:06:58,897::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-926::DEBUG::2014-10-15 07:06:58,898::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:59,094::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:59,095::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:06:59,095::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:06:59,102::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:00,103::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:01,104::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:01,937::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:01,939::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-927::DEBUG::2014-10-15 07:07:01,941::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:01,944::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:01,945::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-928::DEBUG::2014-10-15 07:07:01,947::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-928::DEBUG::2014-10-15 07:07:01,948::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:07:02,106::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:03,107::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-17::DEBUG::2014-10-15 07:07:03,947::task::595::Storage.TaskManager.Task::(_updateState) Task=`31009df7-97cd-4c00-8592-40815fcb5f32`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 07:07:03,947::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=u'486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:07:03,948::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:03,948::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:03,948::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:03,948::__init__::232::IOProcess::(_processLogs) (767) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:07:03,949::__init__::232::IOProcess::(_processLogs) (767) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:03,950::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:03,950::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:03,950::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:03,950::__init__::232::IOProcess::(_processLogs) (768) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:07:03,950::__init__::232::IOProcess::(_processLogs) (768) Queuing response >Thread-17::INFO::2014-10-15 07:07:03,950::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:07:03,951::task::1191::Storage.TaskManager.Task::(prepare) Task=`31009df7-97cd-4c00-8592-40815fcb5f32`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:07:03,951::task::595::Storage.TaskManager.Task::(_updateState) Task=`31009df7-97cd-4c00-8592-40815fcb5f32`::moving from state preparing -> state finished >Thread-17::DEBUG::2014-10-15 07:07:03,951::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 07:07:03,951::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2014-10-15 07:07:03,951::task::993::Storage.TaskManager.Task::(_decref) Task=`31009df7-97cd-4c00-8592-40815fcb5f32`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:07:04,101::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:07:04,101::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:07:04,102::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:07:04,109::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:04,977::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:04,978::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-929::DEBUG::2014-10-15 07:07:04,979::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:04,980::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:04,981::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-930::DEBUG::2014-10-15 07:07:04,982::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-930::DEBUG::2014-10-15 07:07:04,982::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:07:05,110::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:06,111::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:07,112::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:07:07,403::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:07,403::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:07,403::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:07,403::__init__::232::IOProcess::(_processLogs) (769) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:07:07,404::__init__::232::IOProcess::(_processLogs) (769) Queuing response >Thread-25::DEBUG::2014-10-15 07:07:07,404::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:07:07,411::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000485677 s, 945 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:07:07,412::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:07,412::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:07,412::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:07,412::__init__::232::IOProcess::(_processLogs) (770) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:07:07,412::__init__::232::IOProcess::(_processLogs) (770) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:07,413::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:07,413::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:07,413::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:07,413::__init__::232::IOProcess::(_processLogs) (771) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:07,413::__init__::232::IOProcess::(_processLogs) (771) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:07,414::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:07,414::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:07,414::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:07,414::__init__::232::IOProcess::(_processLogs) (772) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:07,414::__init__::232::IOProcess::(_processLogs) (772) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:07,414::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:07,415::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:07,415::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:07,415::__init__::232::IOProcess::(_processLogs) (773) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:07,415::__init__::232::IOProcess::(_processLogs) (773) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:08,004::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:08,005::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-931::DEBUG::2014-10-15 07:07:08,006::task::595::Storage.TaskManager.Task::(_updateState) Task=`23743551-52f9-4561-a372-91c4f0e47046`::moving from state init -> state preparing >Thread-931::INFO::2014-10-15 07:07:08,006::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-931::INFO::2014-10-15 07:07:08,006::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000485677', 'lastCheck': '0.6', 'valid': True}} >Thread-931::DEBUG::2014-10-15 07:07:08,006::task::1191::Storage.TaskManager.Task::(prepare) Task=`23743551-52f9-4561-a372-91c4f0e47046`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000485677', 'lastCheck': '0.6', 'valid': True}} >Thread-931::DEBUG::2014-10-15 07:07:08,006::task::595::Storage.TaskManager.Task::(_updateState) Task=`23743551-52f9-4561-a372-91c4f0e47046`::moving from state preparing -> state finished >Thread-931::DEBUG::2014-10-15 07:07:08,007::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-931::DEBUG::2014-10-15 07:07:08,007::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-931::DEBUG::2014-10-15 07:07:08,007::task::993::Storage.TaskManager.Task::(_decref) Task=`23743551-52f9-4561-a372-91c4f0e47046`::ref 0 aborting False >Thread-931::ERROR::2014-10-15 07:07:08,008::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-931::DEBUG::2014-10-15 07:07:08,008::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:08,019::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:08,019::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-932::DEBUG::2014-10-15 07:07:08,020::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-932::DEBUG::2014-10-15 07:07:08,021::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:07:08,114::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:07:09,108::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:07:09,108::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:07:09,108::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:07:09,115::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:09,115::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:07:09,115::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >VM Channels Listener::DEBUG::2014-10-15 07:07:10,116::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:11,118::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:11,133::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:11,134::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-933::DEBUG::2014-10-15 07:07:11,135::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:11,137::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:11,137::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-934::DEBUG::2014-10-15 07:07:11,138::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-934::DEBUG::2014-10-15 07:07:11,138::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:07:12,119::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:13,120::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:07:14,114::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:07:14,115::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:07:14,115::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:07:14,121::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:14,169::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:14,169::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-935::DEBUG::2014-10-15 07:07:14,170::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:14,171::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:14,172::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-936::DEBUG::2014-10-15 07:07:14,172::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-936::DEBUG::2014-10-15 07:07:14,173::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:07:15,123::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:16,124::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:17,125::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:17,188::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:17,189::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-937::DEBUG::2014-10-15 07:07:17,190::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:17,191::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:17,192::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-938::DEBUG::2014-10-15 07:07:17,193::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-938::DEBUG::2014-10-15 07:07:17,193::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:07:17,416::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:17,416::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:17,416::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:17,416::__init__::232::IOProcess::(_processLogs) (774) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:07:17,416::__init__::232::IOProcess::(_processLogs) (774) Queuing response >Thread-25::DEBUG::2014-10-15 07:07:17,417::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:07:17,423::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000396475 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:07:17,423::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:17,424::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:17,424::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:17,424::__init__::232::IOProcess::(_processLogs) (775) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:07:17,424::__init__::232::IOProcess::(_processLogs) (775) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:17,425::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:17,425::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:17,425::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:17,425::__init__::232::IOProcess::(_processLogs) (776) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:17,425::__init__::232::IOProcess::(_processLogs) (776) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:17,425::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:17,426::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:17,426::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:17,426::__init__::232::IOProcess::(_processLogs) (777) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:17,426::__init__::232::IOProcess::(_processLogs) (777) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:17,426::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:17,426::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:17,426::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:17,426::__init__::232::IOProcess::(_processLogs) (778) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:17,427::__init__::232::IOProcess::(_processLogs) (778) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:07:18,126::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:07:19,121::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:07:19,121::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:07:19,122::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:07:19,128::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:20,129::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:20,212::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:20,213::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-939::DEBUG::2014-10-15 07:07:20,214::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:20,215::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:20,215::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-940::DEBUG::2014-10-15 07:07:20,216::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-940::DEBUG::2014-10-15 07:07:20,217::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:07:21,130::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:22,132::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:23,133::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:23,239::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:23,239::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-941::DEBUG::2014-10-15 07:07:23,240::task::595::Storage.TaskManager.Task::(_updateState) Task=`f9b4da7b-6f75-45e1-8fd1-b174a51f1e71`::moving from state init -> state preparing >Thread-941::INFO::2014-10-15 07:07:23,241::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-941::INFO::2014-10-15 07:07:23,241::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000396475', 'lastCheck': '5.8', 'valid': True}} >Thread-941::DEBUG::2014-10-15 07:07:23,241::task::1191::Storage.TaskManager.Task::(prepare) Task=`f9b4da7b-6f75-45e1-8fd1-b174a51f1e71`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000396475', 'lastCheck': '5.8', 'valid': True}} >Thread-941::DEBUG::2014-10-15 07:07:23,241::task::595::Storage.TaskManager.Task::(_updateState) Task=`f9b4da7b-6f75-45e1-8fd1-b174a51f1e71`::moving from state preparing -> state finished >Thread-941::DEBUG::2014-10-15 07:07:23,241::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-941::DEBUG::2014-10-15 07:07:23,241::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-941::DEBUG::2014-10-15 07:07:23,241::task::993::Storage.TaskManager.Task::(_decref) Task=`f9b4da7b-6f75-45e1-8fd1-b174a51f1e71`::ref 0 aborting False >Thread-941::ERROR::2014-10-15 07:07:23,242::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-941::DEBUG::2014-10-15 07:07:23,243::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:23,251::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:23,252::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-942::DEBUG::2014-10-15 07:07:23,253::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-942::DEBUG::2014-10-15 07:07:23,253::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:07:24,127::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:07:24,128::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:07:24,128::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:07:24,134::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:25,135::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:26,005::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:26,006::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-943::DEBUG::2014-10-15 07:07:26,006::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'VM.migrate' in bridge with {u'params': {u'tunneled': u'false', u'dstqemu': u'10.66.100.111', u'src': u'10.66.100.109', u'dst': u'10.66.100.111:54321', u'vmId': u'c255610f-42c2-426d-9270-e4497f4a0f37', u'abortOnError': u'true', u'method': u'online'}, u'vmID': u'c255610f-42c2-426d-9270-e4497f4a0f37'} >Thread-943::DEBUG::2014-10-15 07:07:26,007::API::510::vds::(migrate) {u'tunneled': u'false', u'dstqemu': u'10.66.100.111', u'src': u'10.66.100.109', u'dst': u'10.66.100.111:54321', u'vmId': u'c255610f-42c2-426d-9270-e4497f4a0f37', u'abortOnError': u'true', u'method': u'online'} >Thread-943::DEBUG::2014-10-15 07:07:26,007::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'VM.migrate' in bridge with True >Thread-943::DEBUG::2014-10-15 07:07:26,008::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-944::DEBUG::2014-10-15 07:07:26,012::migration::102::vm.Vm::(_setupVdsConnection) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Destination server is: 10.66.100.111:54321 >Thread-944::DEBUG::2014-10-15 07:07:26,012::migration::104::vm.Vm::(_setupVdsConnection) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Initiating connection with destination >Thread-944::DEBUG::2014-10-15 07:07:26,021::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-944::DEBUG::2014-10-15 07:07:26,022::migration::154::vm.Vm::(_prepareGuest) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Migration started >Thread-944::DEBUG::2014-10-15 07:07:26,030::migration::237::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::migration semaphore acquired after 0 seconds >VM Channels Listener::DEBUG::2014-10-15 07:07:26,137::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-944::DEBUG::2014-10-15 07:07:26,181::migration::297::vm.Vm::(_startUnderlyingMigration) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::starting migration to qemu+tls://10.66.100.111/system with miguri tcp://10.66.100.111 >Thread-945::DEBUG::2014-10-15 07:07:26,181::migration::360::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::migration downtime thread started >Thread-946::DEBUG::2014-10-15 07:07:26,182::migration::409::vm.Vm::(monitor_migration) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::starting migration monitor thread >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:26,363::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:26,363::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-947::DEBUG::2014-10-15 07:07:26,364::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:26,366::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:26,366::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-948::DEBUG::2014-10-15 07:07:26,367::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-948::DEBUG::2014-10-15 07:07:26,367::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:07:27,138::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:07:27,428::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:27,428::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:27,428::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:27,428::__init__::232::IOProcess::(_processLogs) (779) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:07:27,431::__init__::232::IOProcess::(_processLogs) (779) Queuing response >Thread-25::DEBUG::2014-10-15 07:07:27,431::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:07:27,443::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00308689 s, 149 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:07:27,444::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:27,444::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:27,444::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:27,444::__init__::232::IOProcess::(_processLogs) (780) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:07:27,447::__init__::232::IOProcess::(_processLogs) (780) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:27,447::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:27,447::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:27,448::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:27,448::__init__::232::IOProcess::(_processLogs) (781) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:27,450::__init__::232::IOProcess::(_processLogs) (781) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:27,450::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:27,451::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:27,451::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:27,451::__init__::232::IOProcess::(_processLogs) (782) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:27,454::__init__::232::IOProcess::(_processLogs) (782) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:27,454::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:27,454::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:27,454::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:27,454::__init__::232::IOProcess::(_processLogs) (783) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:27,456::__init__::232::IOProcess::(_processLogs) (783) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:07:28,139::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:29,140::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:29,399::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:29,400::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-949::DEBUG::2014-10-15 07:07:29,401::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:29,402::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:29,403::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-950::DEBUG::2014-10-15 07:07:29,404::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-950::DEBUG::2014-10-15 07:07:29,404::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:07:30,141::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:31,143::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:32,144::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:32,426::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:32,427::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-951::DEBUG::2014-10-15 07:07:32,430::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:32,433::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:32,435::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-952::DEBUG::2014-10-15 07:07:32,436::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-952::DEBUG::2014-10-15 07:07:32,437::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:07:33,146::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:34,147::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:35,148::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:35,494::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:35,495::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-953::DEBUG::2014-10-15 07:07:35,496::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:35,497::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:35,498::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-954::DEBUG::2014-10-15 07:07:35,499::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-954::DEBUG::2014-10-15 07:07:35,499::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:07:36,150::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-946::INFO::2014-10-15 07:07:36,183::migration::466::vm.Vm::(monitor_migration) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Migration Progress: 10 seconds elapsed, 33% of data processed >VM Channels Listener::DEBUG::2014-10-15 07:07:37,151::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:07:37,457::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:37,457::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:37,457::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:37,458::__init__::232::IOProcess::(_processLogs) (784) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:07:37,458::__init__::232::IOProcess::(_processLogs) (784) Queuing response >Thread-25::DEBUG::2014-10-15 07:07:37,459::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:07:37,465::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000336316 s, 1.4 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:07:37,465::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:37,465::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:37,465::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:37,466::__init__::232::IOProcess::(_processLogs) (785) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:07:37,466::__init__::232::IOProcess::(_processLogs) (785) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:37,466::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:37,466::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:37,466::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:37,466::__init__::232::IOProcess::(_processLogs) (786) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:37,467::__init__::232::IOProcess::(_processLogs) (786) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:37,467::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:37,467::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:37,467::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:37,467::__init__::232::IOProcess::(_processLogs) (787) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:37,467::__init__::232::IOProcess::(_processLogs) (787) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:37,468::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:37,468::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:37,468::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:37,468::__init__::232::IOProcess::(_processLogs) (788) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:37,468::__init__::232::IOProcess::(_processLogs) (788) Queuing response >VM Channels Listener::DEBUG::2014-10-15 07:07:38,152::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:38,521::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:38,522::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-955::DEBUG::2014-10-15 07:07:38,523::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d17a5aa-4db6-4733-a5db-2e358d5d4243`::moving from state init -> state preparing >Thread-955::INFO::2014-10-15 07:07:38,523::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-955::INFO::2014-10-15 07:07:38,524::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000336316', 'lastCheck': '1.1', 'valid': True}} >Thread-955::DEBUG::2014-10-15 07:07:38,524::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d17a5aa-4db6-4733-a5db-2e358d5d4243`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000336316', 'lastCheck': '1.1', 'valid': True}} >Thread-955::DEBUG::2014-10-15 07:07:38,524::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d17a5aa-4db6-4733-a5db-2e358d5d4243`::moving from state preparing -> state finished >Thread-955::DEBUG::2014-10-15 07:07:38,524::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-955::DEBUG::2014-10-15 07:07:38,524::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-955::DEBUG::2014-10-15 07:07:38,524::task::993::Storage.TaskManager.Task::(_decref) Task=`1d17a5aa-4db6-4733-a5db-2e358d5d4243`::ref 0 aborting False >Thread-955::ERROR::2014-10-15 07:07:38,525::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-955::DEBUG::2014-10-15 07:07:38,526::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:38,545::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:38,545::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-956::DEBUG::2014-10-15 07:07:38,546::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-956::DEBUG::2014-10-15 07:07:38,547::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:07:39,154::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:39,154::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:07:39,154::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 11 >VM Channels Listener::DEBUG::2014-10-15 07:07:40,156::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:41,157::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-945::DEBUG::2014-10-15 07:07:41,182::migration::369::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::setting migration downtime to 50 >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:41,670::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:41,670::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-957::DEBUG::2014-10-15 07:07:41,672::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:41,673::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:41,673::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-958::DEBUG::2014-10-15 07:07:41,674::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-958::DEBUG::2014-10-15 07:07:41,674::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:07:42,158::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:43,160::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:44,161::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:44,991::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:44,992::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-959::DEBUG::2014-10-15 07:07:44,993::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:44,994::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:44,995::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-960::DEBUG::2014-10-15 07:07:44,996::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-960::DEBUG::2014-10-15 07:07:44,996::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:07:45,162::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:46,164::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-946::INFO::2014-10-15 07:07:46,184::migration::466::vm.Vm::(monitor_migration) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Migration Progress: 20 seconds elapsed, 84% of data processed >VM Channels Listener::DEBUG::2014-10-15 07:07:47,165::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-16::DEBUG::2014-10-15 07:07:47,469::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:47,469::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:47,469::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:47,470::__init__::232::IOProcess::(_processLogs) (789) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:07:47,471::__init__::232::IOProcess::(_processLogs) (789) Queuing response >Thread-25::DEBUG::2014-10-15 07:07:47,471::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:07:47,477::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000375715 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:07:47,478::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:47,478::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:47,478::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:47,478::__init__::232::IOProcess::(_processLogs) (790) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:07:47,478::__init__::232::IOProcess::(_processLogs) (790) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:47,479::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:47,479::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:47,479::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:47,479::__init__::232::IOProcess::(_processLogs) (791) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:47,479::__init__::232::IOProcess::(_processLogs) (791) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:47,480::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:47,480::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:47,480::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:47,480::__init__::232::IOProcess::(_processLogs) (792) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:47,480::__init__::232::IOProcess::(_processLogs) (792) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:47,480::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:47,480::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:47,481::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:47,481::__init__::232::IOProcess::(_processLogs) (793) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:47,481::__init__::232::IOProcess::(_processLogs) (793) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:48,016::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:48,017::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-961::DEBUG::2014-10-15 07:07:48,018::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:48,020::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:48,020::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-962::DEBUG::2014-10-15 07:07:48,021::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-962::DEBUG::2014-10-15 07:07:48,021::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:07:48,166::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:49,168::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:50,169::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:51,041::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:51,042::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-963::DEBUG::2014-10-15 07:07:51,043::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:51,044::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:51,045::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-964::DEBUG::2014-10-15 07:07:51,045::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-964::DEBUG::2014-10-15 07:07:51,046::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:07:51,171::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:52,172::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >VM Channels Listener::DEBUG::2014-10-15 07:07:53,173::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >libvirtEventLoop::DEBUG::2014-10-15 07:07:53,198::vm::5461::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::event Suspended detail 0 opaque None >libvirtEventLoop::DEBUG::2014-10-15 07:07:53,748::vm::5461::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::event Stopped detail 3 opaque None >Thread-944::DEBUG::2014-10-15 07:07:53,749::migration::375::vm.Vm::(cancel) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::canceling migration downtime thread >Thread-944::DEBUG::2014-10-15 07:07:53,749::migration::469::vm.Vm::(stop) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::stopping migration monitor thread >Thread-945::DEBUG::2014-10-15 07:07:53,749::migration::372::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::migration downtime thread exiting >Thread-946::DEBUG::2014-10-15 07:07:53,750::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 42 edom: 10 level: 2 message: Domain not found: no domain with matching uuid 'c255610f-42c2-426d-9270-e4497f4a0f37' (rhevh-guest-rhel7.0) >Thread-944::DEBUG::2014-10-15 07:07:53,753::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 42 edom: 10 level: 2 message: Domain not found: no domain with matching uuid 'c255610f-42c2-426d-9270-e4497f4a0f37' (rhevh-guest-rhel7.0) >Thread-944::DEBUG::2014-10-15 07:07:53,753::vm::2838::vm.Vm::(setDownStatus) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Changed state to Down: Migration succeeded (code=4) >Thread-944::DEBUG::2014-10-15 07:07:53,753::vmchannels::210::vds::(unregister) Delete fileno 30 from listener. >Thread-944::DEBUG::2014-10-15 07:07:53,754::sampling::412::vm.Vm::(stop) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stop statistics collection >Thread-17::DEBUG::2014-10-15 07:07:53,754::sampling::443::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stats thread finished >Thread-944::DEBUG::2014-10-15 07:07:53,756::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 42 edom: 10 level: 2 message: Domain not found: no domain with matching uuid 'c255610f-42c2-426d-9270-e4497f4a0f37' (rhevh-guest-rhel7.0) >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:54,064::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:54,064::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-965::DEBUG::2014-10-15 07:07:54,065::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d908070-2e40-4e4a-b874-9215be6e5bbb`::moving from state init -> state preparing >Thread-965::INFO::2014-10-15 07:07:54,066::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-965::INFO::2014-10-15 07:07:54,066::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000375715', 'lastCheck': '6.6', 'valid': True}} >Thread-965::DEBUG::2014-10-15 07:07:54,066::task::1191::Storage.TaskManager.Task::(prepare) Task=`4d908070-2e40-4e4a-b874-9215be6e5bbb`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000375715', 'lastCheck': '6.6', 'valid': True}} >Thread-965::DEBUG::2014-10-15 07:07:54,066::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d908070-2e40-4e4a-b874-9215be6e5bbb`::moving from state preparing -> state finished >Thread-965::DEBUG::2014-10-15 07:07:54,066::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-965::DEBUG::2014-10-15 07:07:54,066::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-965::DEBUG::2014-10-15 07:07:54,066::task::993::Storage.TaskManager.Task::(_decref) Task=`4d908070-2e40-4e4a-b874-9215be6e5bbb`::ref 0 aborting False >Thread-965::ERROR::2014-10-15 07:07:54,067::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-965::DEBUG::2014-10-15 07:07:54,068::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:54,080::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:54,081::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-966::DEBUG::2014-10-15 07:07:54,082::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:54,085::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:54,085::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-967::DEBUG::2014-10-15 07:07:54,085::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'VM.destroy' in bridge with {u'vmID': u'c255610f-42c2-426d-9270-e4497f4a0f37'} >Thread-967::INFO::2014-10-15 07:07:54,086::API::332::vds::(destroy) vmContainerLock acquired by vm c255610f-42c2-426d-9270-e4497f4a0f37 >Thread-967::DEBUG::2014-10-15 07:07:54,086::vm::4919::vm.Vm::(destroy) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::destroy Called >Thread-967::INFO::2014-10-15 07:07:54,086::vm::4845::vm.Vm::(releaseVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Release VM resources >Thread-967::WARNING::2014-10-15 07:07:54,086::vm::1955::vm.Vm::(_set_lastStatus) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::trying to set state to Powering down when already Down >Thread-967::DEBUG::2014-10-15 07:07:54,086::sampling::412::vm.Vm::(stop) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stop statistics collection >Thread-967::DEBUG::2014-10-15 07:07:54,087::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 42 edom: 10 level: 2 message: Domain not found: no domain with matching uuid 'c255610f-42c2-426d-9270-e4497f4a0f37' (rhevh-guest-rhel7.0) >Thread-967::INFO::2014-10-15 07:07:54,087::vm::4888::vm.Vm::(_destroyVmGraceful) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::VM 'c255610f-42c2-426d-9270-e4497f4a0f37' already down and destroyed >Thread-967::DEBUG::2014-10-15 07:07:54,087::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6ed8c1f-3fbb-43d6-bad8-1229ad847799`::moving from state init -> state preparing >Thread-967::INFO::2014-10-15 07:07:54,087::logUtils::44::dispatcher::(wrapper) Run and protect: teardownImage(sdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID=u'00000002-0002-0002-0002-0000000003e1', imgUUID=u'a5611808-d282-45f9-9771-c3797b521610', volUUID=None) >Thread-967::DEBUG::2014-10-15 07:07:54,088::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4`ReqID=`86a8752f-fad2-4f07-8e9a-67e75dc54519`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '3262' at 'teardownImage' >Thread-967::DEBUG::2014-10-15 07:07:54,088::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' for lock type 'shared' >Thread-967::DEBUG::2014-10-15 07:07:54,088::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' is free. Now locking as 'shared' (1 active user) >Thread-967::DEBUG::2014-10-15 07:07:54,088::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4`ReqID=`86a8752f-fad2-4f07-8e9a-67e75dc54519`::Granted request >Thread-967::DEBUG::2014-10-15 07:07:54,088::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e6ed8c1f-3fbb-43d6-bad8-1229ad847799`::_resourcesAcquired: Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 (shared) >Thread-967::DEBUG::2014-10-15 07:07:54,088::task::993::Storage.TaskManager.Task::(_decref) Task=`e6ed8c1f-3fbb-43d6-bad8-1229ad847799`::ref 1 aborting False >Thread-967::INFO::2014-10-15 07:07:54,089::logUtils::47::dispatcher::(wrapper) Run and protect: teardownImage, Return response: None >Thread-967::DEBUG::2014-10-15 07:07:54,089::task::1191::Storage.TaskManager.Task::(prepare) Task=`e6ed8c1f-3fbb-43d6-bad8-1229ad847799`::finished: None >Thread-967::DEBUG::2014-10-15 07:07:54,089::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6ed8c1f-3fbb-43d6-bad8-1229ad847799`::moving from state preparing -> state finished >Thread-967::DEBUG::2014-10-15 07:07:54,089::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': < ResourceRef 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', isValid: 'True' obj: 'None'>} >Thread-967::DEBUG::2014-10-15 07:07:54,089::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-967::DEBUG::2014-10-15 07:07:54,089::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' >Thread-967::DEBUG::2014-10-15 07:07:54,089::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' (0 active users) >Thread-967::DEBUG::2014-10-15 07:07:54,089::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' is free, finding out if anyone is waiting for it. >Thread-967::DEBUG::2014-10-15 07:07:54,089::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', Clearing records. >Thread-967::DEBUG::2014-10-15 07:07:54,090::task::993::Storage.TaskManager.Task::(_decref) Task=`e6ed8c1f-3fbb-43d6-bad8-1229ad847799`::ref 0 aborting False >Thread-967::DEBUG::2014-10-15 07:07:54,090::task::595::Storage.TaskManager.Task::(_updateState) Task=`a1bee9fc-0e62-40dd-8601-b745a3cbb6b8`::moving from state init -> state preparing >Thread-967::INFO::2014-10-15 07:07:54,090::logUtils::44::dispatcher::(wrapper) Run and protect: inappropriateDevices(thiefId=u'c255610f-42c2-426d-9270-e4497f4a0f37') >Thread-967::INFO::2014-10-15 07:07:54,091::logUtils::47::dispatcher::(wrapper) Run and protect: inappropriateDevices, Return response: None >Thread-967::DEBUG::2014-10-15 07:07:54,091::task::1191::Storage.TaskManager.Task::(prepare) Task=`a1bee9fc-0e62-40dd-8601-b745a3cbb6b8`::finished: None >Thread-967::DEBUG::2014-10-15 07:07:54,091::task::595::Storage.TaskManager.Task::(_updateState) Task=`a1bee9fc-0e62-40dd-8601-b745a3cbb6b8`::moving from state preparing -> state finished >Thread-967::DEBUG::2014-10-15 07:07:54,092::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-967::DEBUG::2014-10-15 07:07:54,092::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-967::DEBUG::2014-10-15 07:07:54,092::task::993::Storage.TaskManager.Task::(_decref) Task=`a1bee9fc-0e62-40dd-8601-b745a3cbb6b8`::ref 0 aborting False >Thread-967::DEBUG::2014-10-15 07:07:54,092::utils::738::root::(execCmd) /usr/libexec/vdsm/hooks/after_vm_destroy/50_vhostmd (cwd None) >VM Channels Listener::DEBUG::2014-10-15 07:07:54,174::vmchannels::117::vds::(_do_del_channels) fileno 30 was removed from listener. >Detector thread::DEBUG::2014-10-15 07:07:54,176::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 127.0.0.1:55087 >Detector thread::DEBUG::2014-10-15 07:07:54,176::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 127.0.0.1:55087 >Detector thread::DEBUG::2014-10-15 07:07:54,177::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 127.0.0.1:55087 >Detector thread::DEBUG::2014-10-15 07:07:54,177::BindingXMLRPC::1172::XmlDetector::(handleSocket) xml over http detected from ('127.0.0.1', 55087) >Thread-967::DEBUG::2014-10-15 07:07:54,226::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-967::INFO::2014-10-15 07:07:54,227::hooks::100::root::(_runHooksDir) >Thread-967::DEBUG::2014-10-15 07:07:54,227::vm::4913::vm.Vm::(deleteVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Total desktops after destroy of c255610f-42c2-426d-9270-e4497f4a0f37 is 0 >Thread-967::DEBUG::2014-10-15 07:07:54,227::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'VM.destroy' in bridge with True >Thread-967::DEBUG::2014-10-15 07:07:54,227::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:54,234::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:54,235::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-969::DEBUG::2014-10-15 07:07:54,235::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getVMFullList' in bridge with {u'vmList': [u'c255610f-42c2-426d-9270-e4497f4a0f37']} >Thread-969::DEBUG::2014-10-15 07:07:54,236::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getVMFullList' in bridge with [] >Thread-969::DEBUG::2014-10-15 07:07:54,236::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:07:57,349::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:07:57,350::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-970::DEBUG::2014-10-15 07:07:57,351::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:07:57,482::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:57,482::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:57,482::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:57,482::__init__::232::IOProcess::(_processLogs) (794) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:07:57,482::__init__::232::IOProcess::(_processLogs) (794) Queuing response >Thread-25::DEBUG::2014-10-15 07:07:57,483::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:07:57,489::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000369074 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:07:57,490::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:57,490::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:57,490::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:57,490::__init__::232::IOProcess::(_processLogs) (795) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:07:57,490::__init__::232::IOProcess::(_processLogs) (795) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:57,491::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:57,491::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:57,491::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:57,491::__init__::232::IOProcess::(_processLogs) (796) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:57,491::__init__::232::IOProcess::(_processLogs) (796) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:57,492::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:57,492::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:57,492::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:57,492::__init__::232::IOProcess::(_processLogs) (797) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:57,492::__init__::232::IOProcess::(_processLogs) (797) Queuing response >Thread-16::DEBUG::2014-10-15 07:07:57,493::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:07:57,493::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:07:57,493::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:07:57,493::__init__::232::IOProcess::(_processLogs) (798) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:07:57,493::__init__::232::IOProcess::(_processLogs) (798) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:00,366::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:00,367::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-971::DEBUG::2014-10-15 07:08:00,368::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:03,405::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:03,405::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-972::DEBUG::2014-10-15 07:08:03,406::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:06,434::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:06,435::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-973::DEBUG::2014-10-15 07:08:06,436::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:08:07,494::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:07,495::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:07,495::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:07,495::__init__::232::IOProcess::(_processLogs) (799) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:08:07,495::__init__::232::IOProcess::(_processLogs) (799) Queuing response >Thread-25::DEBUG::2014-10-15 07:08:07,496::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:08:07,503::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000482468 s, 951 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:08:07,504::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:07,504::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:07,504::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:07,504::__init__::232::IOProcess::(_processLogs) (800) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:08:07,504::__init__::232::IOProcess::(_processLogs) (800) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:07,505::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:07,505::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:07,505::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:07,505::__init__::232::IOProcess::(_processLogs) (801) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:07,505::__init__::232::IOProcess::(_processLogs) (801) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:07,505::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:07,505::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:07,506::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:07,506::__init__::232::IOProcess::(_processLogs) (802) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:07,506::__init__::232::IOProcess::(_processLogs) (802) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:07,506::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:07,506::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:07,506::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:07,506::__init__::232::IOProcess::(_processLogs) (803) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:07,506::__init__::232::IOProcess::(_processLogs) (803) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:09,461::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:09,462::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-974::DEBUG::2014-10-15 07:08:09,463::task::595::Storage.TaskManager.Task::(_updateState) Task=`f144c2a8-bcae-4326-92ea-7ea40a5d8c22`::moving from state init -> state preparing >Thread-974::INFO::2014-10-15 07:08:09,463::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-974::INFO::2014-10-15 07:08:09,463::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000482468', 'lastCheck': '2.0', 'valid': True}} >Thread-974::DEBUG::2014-10-15 07:08:09,463::task::1191::Storage.TaskManager.Task::(prepare) Task=`f144c2a8-bcae-4326-92ea-7ea40a5d8c22`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000482468', 'lastCheck': '2.0', 'valid': True}} >Thread-974::DEBUG::2014-10-15 07:08:09,464::task::595::Storage.TaskManager.Task::(_updateState) Task=`f144c2a8-bcae-4326-92ea-7ea40a5d8c22`::moving from state preparing -> state finished >Thread-974::DEBUG::2014-10-15 07:08:09,464::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-974::DEBUG::2014-10-15 07:08:09,464::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-974::DEBUG::2014-10-15 07:08:09,464::task::993::Storage.TaskManager.Task::(_decref) Task=`f144c2a8-bcae-4326-92ea-7ea40a5d8c22`::ref 0 aborting False >Thread-974::ERROR::2014-10-15 07:08:09,465::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-974::DEBUG::2014-10-15 07:08:09,465::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:09,487::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:09,488::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-975::DEBUG::2014-10-15 07:08:09,489::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:12,567::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:12,568::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-976::DEBUG::2014-10-15 07:08:12,569::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:15,579::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:15,579::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-977::DEBUG::2014-10-15 07:08:15,580::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:08:17,508::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:17,508::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:17,508::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:17,508::__init__::232::IOProcess::(_processLogs) (804) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:08:17,508::__init__::232::IOProcess::(_processLogs) (804) Queuing response >Thread-25::DEBUG::2014-10-15 07:08:17,509::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:08:17,515::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000446385 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:08:17,515::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:17,515::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:17,516::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:17,516::__init__::232::IOProcess::(_processLogs) (805) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:08:17,516::__init__::232::IOProcess::(_processLogs) (805) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:17,516::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:17,517::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:17,517::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:17,517::__init__::232::IOProcess::(_processLogs) (806) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:17,517::__init__::232::IOProcess::(_processLogs) (806) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:17,517::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:17,517::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:17,517::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:17,518::__init__::232::IOProcess::(_processLogs) (807) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:17,518::__init__::232::IOProcess::(_processLogs) (807) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:17,518::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:17,518::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:17,518::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:17,518::__init__::232::IOProcess::(_processLogs) (808) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:17,518::__init__::232::IOProcess::(_processLogs) (808) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:18,589::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:18,590::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-978::DEBUG::2014-10-15 07:08:18,591::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:21,602::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:21,603::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-979::DEBUG::2014-10-15 07:08:21,604::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:24,614::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:24,615::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-980::DEBUG::2014-10-15 07:08:24,616::task::595::Storage.TaskManager.Task::(_updateState) Task=`101248ea-5783-4699-acd8-355f1a55a919`::moving from state init -> state preparing >Thread-980::INFO::2014-10-15 07:08:24,616::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-980::INFO::2014-10-15 07:08:24,616::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000446385', 'lastCheck': '7.1', 'valid': True}} >Thread-980::DEBUG::2014-10-15 07:08:24,616::task::1191::Storage.TaskManager.Task::(prepare) Task=`101248ea-5783-4699-acd8-355f1a55a919`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000446385', 'lastCheck': '7.1', 'valid': True}} >Thread-980::DEBUG::2014-10-15 07:08:24,616::task::595::Storage.TaskManager.Task::(_updateState) Task=`101248ea-5783-4699-acd8-355f1a55a919`::moving from state preparing -> state finished >Thread-980::DEBUG::2014-10-15 07:08:24,616::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-980::DEBUG::2014-10-15 07:08:24,616::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-980::DEBUG::2014-10-15 07:08:24,617::task::993::Storage.TaskManager.Task::(_decref) Task=`101248ea-5783-4699-acd8-355f1a55a919`::ref 0 aborting False >Thread-980::ERROR::2014-10-15 07:08:24,617::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-980::DEBUG::2014-10-15 07:08:24,618::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:24,633::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:24,634::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-981::DEBUG::2014-10-15 07:08:24,635::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:08:27,519::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:27,520::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:27,520::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:27,520::__init__::232::IOProcess::(_processLogs) (809) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:08:27,520::__init__::232::IOProcess::(_processLogs) (809) Queuing response >Thread-25::DEBUG::2014-10-15 07:08:27,521::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:08:27,527::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00035871 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:08:27,527::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:27,527::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:27,528::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:27,528::__init__::232::IOProcess::(_processLogs) (810) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:08:27,528::__init__::232::IOProcess::(_processLogs) (810) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:27,528::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:27,529::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:27,529::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:27,529::__init__::232::IOProcess::(_processLogs) (811) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:27,529::__init__::232::IOProcess::(_processLogs) (811) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:27,529::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:27,529::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:27,529::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:27,530::__init__::232::IOProcess::(_processLogs) (812) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:27,530::__init__::232::IOProcess::(_processLogs) (812) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:27,530::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:27,530::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:27,530::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:27,530::__init__::232::IOProcess::(_processLogs) (813) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:27,530::__init__::232::IOProcess::(_processLogs) (813) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:27,693::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:27,694::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-982::DEBUG::2014-10-15 07:08:27,695::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:30,707::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:30,707::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-983::DEBUG::2014-10-15 07:08:30,708::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:33,720::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:33,720::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-984::DEBUG::2014-10-15 07:08:33,721::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:36,729::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:36,730::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-985::DEBUG::2014-10-15 07:08:36,731::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:08:37,531::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:37,532::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:37,532::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:37,532::__init__::232::IOProcess::(_processLogs) (814) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:08:37,532::__init__::232::IOProcess::(_processLogs) (814) Queuing response >Thread-25::DEBUG::2014-10-15 07:08:37,532::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:08:37,538::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00042339 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:08:37,539::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:37,539::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:37,539::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:37,539::__init__::232::IOProcess::(_processLogs) (815) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:08:37,539::__init__::232::IOProcess::(_processLogs) (815) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:37,540::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:37,540::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:37,540::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:37,540::__init__::232::IOProcess::(_processLogs) (816) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:37,540::__init__::232::IOProcess::(_processLogs) (816) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:37,541::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:37,541::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:37,541::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:37,541::__init__::232::IOProcess::(_processLogs) (817) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:37,541::__init__::232::IOProcess::(_processLogs) (817) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:37,541::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:37,542::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:37,542::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:37,542::__init__::232::IOProcess::(_processLogs) (818) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:37,542::__init__::232::IOProcess::(_processLogs) (818) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:39,739::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:39,740::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-986::DEBUG::2014-10-15 07:08:39,741::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ebf8967-9df7-43c4-81d8-5a84b4e8f68d`::moving from state init -> state preparing >Thread-986::INFO::2014-10-15 07:08:39,741::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-986::INFO::2014-10-15 07:08:39,741::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00042339', 'lastCheck': '2.2', 'valid': True}} >Thread-986::DEBUG::2014-10-15 07:08:39,742::task::1191::Storage.TaskManager.Task::(prepare) Task=`7ebf8967-9df7-43c4-81d8-5a84b4e8f68d`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00042339', 'lastCheck': '2.2', 'valid': True}} >Thread-986::DEBUG::2014-10-15 07:08:39,742::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ebf8967-9df7-43c4-81d8-5a84b4e8f68d`::moving from state preparing -> state finished >Thread-986::DEBUG::2014-10-15 07:08:39,742::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-986::DEBUG::2014-10-15 07:08:39,742::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-986::DEBUG::2014-10-15 07:08:39,742::task::993::Storage.TaskManager.Task::(_decref) Task=`7ebf8967-9df7-43c4-81d8-5a84b4e8f68d`::ref 0 aborting False >Thread-986::ERROR::2014-10-15 07:08:39,743::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-986::DEBUG::2014-10-15 07:08:39,743::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:39,755::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:39,756::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-987::DEBUG::2014-10-15 07:08:39,756::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:42,825::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:42,826::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-988::DEBUG::2014-10-15 07:08:42,827::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:45,838::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:45,839::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-989::DEBUG::2014-10-15 07:08:45,840::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:08:47,543::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:47,543::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:47,543::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:47,543::__init__::232::IOProcess::(_processLogs) (819) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:08:47,543::__init__::232::IOProcess::(_processLogs) (819) Queuing response >Thread-25::DEBUG::2014-10-15 07:08:47,544::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:08:47,550::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000426419 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:08:47,550::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:47,551::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:47,551::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:47,551::__init__::232::IOProcess::(_processLogs) (820) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:08:47,551::__init__::232::IOProcess::(_processLogs) (820) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:47,552::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:47,552::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:47,552::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:47,552::__init__::232::IOProcess::(_processLogs) (821) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:47,552::__init__::232::IOProcess::(_processLogs) (821) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:47,552::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:47,552::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:47,552::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:47,553::__init__::232::IOProcess::(_processLogs) (822) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:47,553::__init__::232::IOProcess::(_processLogs) (822) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:47,553::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:47,553::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:47,553::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:47,553::__init__::232::IOProcess::(_processLogs) (823) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:47,553::__init__::232::IOProcess::(_processLogs) (823) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:48,850::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:48,851::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-990::DEBUG::2014-10-15 07:08:48,852::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:51,862::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:51,863::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-991::DEBUG::2014-10-15 07:08:51,864::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:54,874::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:54,874::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-992::DEBUG::2014-10-15 07:08:54,876::task::595::Storage.TaskManager.Task::(_updateState) Task=`4daa6f56-fcc7-4cdb-9946-e2d7455907ff`::moving from state init -> state preparing >Thread-992::INFO::2014-10-15 07:08:54,876::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-992::INFO::2014-10-15 07:08:54,876::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426419', 'lastCheck': '7.3', 'valid': True}} >Thread-992::DEBUG::2014-10-15 07:08:54,876::task::1191::Storage.TaskManager.Task::(prepare) Task=`4daa6f56-fcc7-4cdb-9946-e2d7455907ff`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426419', 'lastCheck': '7.3', 'valid': True}} >Thread-992::DEBUG::2014-10-15 07:08:54,876::task::595::Storage.TaskManager.Task::(_updateState) Task=`4daa6f56-fcc7-4cdb-9946-e2d7455907ff`::moving from state preparing -> state finished >Thread-992::DEBUG::2014-10-15 07:08:54,876::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-992::DEBUG::2014-10-15 07:08:54,876::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-992::DEBUG::2014-10-15 07:08:54,876::task::993::Storage.TaskManager.Task::(_decref) Task=`4daa6f56-fcc7-4cdb-9946-e2d7455907ff`::ref 0 aborting False >Thread-992::ERROR::2014-10-15 07:08:54,877::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-992::DEBUG::2014-10-15 07:08:54,878::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:54,888::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:54,888::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-993::DEBUG::2014-10-15 07:08:54,889::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:08:57,554::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:57,555::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:57,555::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:57,555::__init__::232::IOProcess::(_processLogs) (824) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:08:57,555::__init__::232::IOProcess::(_processLogs) (824) Queuing response >Thread-25::DEBUG::2014-10-15 07:08:57,555::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:08:57,561::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000405748 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:08:57,562::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:57,562::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:57,562::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:57,562::__init__::232::IOProcess::(_processLogs) (825) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:08:57,562::__init__::232::IOProcess::(_processLogs) (825) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:57,563::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:57,563::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:57,563::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:57,563::__init__::232::IOProcess::(_processLogs) (826) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:57,563::__init__::232::IOProcess::(_processLogs) (826) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:57,564::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:57,564::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:57,564::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:57,564::__init__::232::IOProcess::(_processLogs) (827) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:57,564::__init__::232::IOProcess::(_processLogs) (827) Queuing response >Thread-16::DEBUG::2014-10-15 07:08:57,565::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:08:57,565::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:08:57,565::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:08:57,565::__init__::232::IOProcess::(_processLogs) (828) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:08:57,565::__init__::232::IOProcess::(_processLogs) (828) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:08:57,981::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:08:57,982::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-994::DEBUG::2014-10-15 07:08:57,983::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:00,994::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:00,995::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-995::DEBUG::2014-10-15 07:09:00,996::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:04,006::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:04,007::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-996::DEBUG::2014-10-15 07:09:04,008::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:07,018::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:07,019::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-997::DEBUG::2014-10-15 07:09:07,020::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:09:07,566::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:07,566::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:07,566::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:07,566::__init__::232::IOProcess::(_processLogs) (829) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:09:07,567::__init__::232::IOProcess::(_processLogs) (829) Queuing response >Thread-25::DEBUG::2014-10-15 07:09:07,567::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:09:07,574::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00044019 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:09:07,575::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:07,575::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:07,575::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:07,575::__init__::232::IOProcess::(_processLogs) (830) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:09:07,575::__init__::232::IOProcess::(_processLogs) (830) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:07,576::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:07,576::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:07,576::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:07,576::__init__::232::IOProcess::(_processLogs) (831) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:07,576::__init__::232::IOProcess::(_processLogs) (831) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:07,577::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:07,577::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:07,577::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:07,577::__init__::232::IOProcess::(_processLogs) (832) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:07,577::__init__::232::IOProcess::(_processLogs) (832) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:07,578::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:07,578::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:07,578::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:07,578::__init__::232::IOProcess::(_processLogs) (833) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:07,578::__init__::232::IOProcess::(_processLogs) (833) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:10,030::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:10,031::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-998::DEBUG::2014-10-15 07:09:10,032::task::595::Storage.TaskManager.Task::(_updateState) Task=`15d29db2-20b1-4f2d-ad2c-153469c1be96`::moving from state init -> state preparing >Thread-998::INFO::2014-10-15 07:09:10,032::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-998::INFO::2014-10-15 07:09:10,033::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00044019', 'lastCheck': '2.5', 'valid': True}} >Thread-998::DEBUG::2014-10-15 07:09:10,033::task::1191::Storage.TaskManager.Task::(prepare) Task=`15d29db2-20b1-4f2d-ad2c-153469c1be96`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00044019', 'lastCheck': '2.5', 'valid': True}} >Thread-998::DEBUG::2014-10-15 07:09:10,033::task::595::Storage.TaskManager.Task::(_updateState) Task=`15d29db2-20b1-4f2d-ad2c-153469c1be96`::moving from state preparing -> state finished >Thread-998::DEBUG::2014-10-15 07:09:10,033::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-998::DEBUG::2014-10-15 07:09:10,033::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-998::DEBUG::2014-10-15 07:09:10,033::task::993::Storage.TaskManager.Task::(_decref) Task=`15d29db2-20b1-4f2d-ad2c-153469c1be96`::ref 0 aborting False >Thread-998::ERROR::2014-10-15 07:09:10,034::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-998::DEBUG::2014-10-15 07:09:10,035::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:10,047::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:10,048::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-999::DEBUG::2014-10-15 07:09:10,050::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:13,125::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:13,125::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1000::DEBUG::2014-10-15 07:09:13,126::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:16,136::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:16,137::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1001::DEBUG::2014-10-15 07:09:16,138::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:09:17,579::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:17,579::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:17,579::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:17,579::__init__::232::IOProcess::(_processLogs) (834) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:09:17,579::__init__::232::IOProcess::(_processLogs) (834) Queuing response >Thread-25::DEBUG::2014-10-15 07:09:17,580::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:09:17,586::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000508983 s, 902 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:09:17,587::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:17,587::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:17,587::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:17,587::__init__::232::IOProcess::(_processLogs) (835) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:09:17,588::__init__::232::IOProcess::(_processLogs) (835) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:17,588::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:17,588::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:17,588::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:17,588::__init__::232::IOProcess::(_processLogs) (836) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:17,589::__init__::232::IOProcess::(_processLogs) (836) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:17,589::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:17,589::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:17,589::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:17,589::__init__::232::IOProcess::(_processLogs) (837) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:17,589::__init__::232::IOProcess::(_processLogs) (837) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:17,590::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:17,590::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:17,590::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:17,590::__init__::232::IOProcess::(_processLogs) (838) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:17,590::__init__::232::IOProcess::(_processLogs) (838) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:19,148::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:19,148::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1002::DEBUG::2014-10-15 07:09:19,150::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:22,158::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:22,159::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1003::DEBUG::2014-10-15 07:09:22,160::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:25,170::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:25,171::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1004::DEBUG::2014-10-15 07:09:25,172::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc2921d1-d31c-4654-8999-05342e5e5b59`::moving from state init -> state preparing >Thread-1004::INFO::2014-10-15 07:09:25,172::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1004::INFO::2014-10-15 07:09:25,172::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000508983', 'lastCheck': '7.6', 'valid': True}} >Thread-1004::DEBUG::2014-10-15 07:09:25,173::task::1191::Storage.TaskManager.Task::(prepare) Task=`cc2921d1-d31c-4654-8999-05342e5e5b59`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000508983', 'lastCheck': '7.6', 'valid': True}} >Thread-1004::DEBUG::2014-10-15 07:09:25,173::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc2921d1-d31c-4654-8999-05342e5e5b59`::moving from state preparing -> state finished >Thread-1004::DEBUG::2014-10-15 07:09:25,173::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1004::DEBUG::2014-10-15 07:09:25,173::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1004::DEBUG::2014-10-15 07:09:25,173::task::993::Storage.TaskManager.Task::(_decref) Task=`cc2921d1-d31c-4654-8999-05342e5e5b59`::ref 0 aborting False >Thread-1004::ERROR::2014-10-15 07:09:25,174::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1004::DEBUG::2014-10-15 07:09:25,174::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:25,183::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:25,183::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1005::DEBUG::2014-10-15 07:09:25,184::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-25::DEBUG::2014-10-15 07:09:27,591::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 07:09:27,591::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:27,592::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:27,592::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:27,592::__init__::232::IOProcess::(_processLogs) (839) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:09:27,592::__init__::232::IOProcess::(_processLogs) (839) Queuing response >Thread-25::DEBUG::2014-10-15 07:09:27,599::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 07:09:27,599::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:27,599::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:27,599::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:27,600::__init__::232::IOProcess::(_processLogs) (840) Got request for method 'touch' >Thread-16::DEBUG::2014-10-15 07:09:27,640::__init__::232::IOProcess::(_processLogs) (840) Queuing response >Thread-25::DEBUG::2014-10-15 07:09:27,640::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-16::DEBUG::2014-10-15 07:09:27,641::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:27,641::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:27,641::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:27,641::__init__::232::IOProcess::(_processLogs) (841) Got request for method 'readfile' >Thread-16::DEBUG::2014-10-15 07:09:27,641::__init__::232::IOProcess::(_processLogs) (841) Queuing response >Thread-25::DEBUG::2014-10-15 07:09:27,642::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=domain1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-0000000003e1', 'REMOTE_PATH=10.66.128.104:/home/NFS1', 'ROLE=Master', 'SDUUID=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b90f33781d11f6e6b0b3945de3167c0719a458c1'] >Thread-16::DEBUG::2014-10-15 07:09:27,642::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:27,642::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:27,642::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:27,642::__init__::232::IOProcess::(_processLogs) (842) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:27,643::__init__::232::IOProcess::(_processLogs) (842) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:27,643::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:27,643::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:27,643::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:27,643::__init__::232::IOProcess::(_processLogs) (843) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:09:27,644::__init__::232::IOProcess::(_processLogs) (843) Queuing response >Thread-25::DEBUG::2014-10-15 07:09:27,644::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-25::INFO::2014-10-15 07:09:27,644::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_imageNS already registered >Thread-25::INFO::2014-10-15 07:09:27,644::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_volumeNS already registered >Thread-16::DEBUG::2014-10-15 07:09:27,644::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:27,645::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:27,645::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:27,645::__init__::232::IOProcess::(_processLogs) (844) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:09:27,645::__init__::232::IOProcess::(_processLogs) (844) Queuing response >Thread-25::DEBUG::2014-10-15 07:09:27,645::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:09:27,651::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000370901 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:09:27,652::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:27,652::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:27,652::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:27,652::__init__::232::IOProcess::(_processLogs) (845) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:09:27,652::__init__::232::IOProcess::(_processLogs) (845) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:27,653::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:27,653::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:27,653::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:27,653::__init__::232::IOProcess::(_processLogs) (846) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:27,653::__init__::232::IOProcess::(_processLogs) (846) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:27,654::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:27,654::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:27,654::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:27,654::__init__::232::IOProcess::(_processLogs) (847) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:27,654::__init__::232::IOProcess::(_processLogs) (847) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:27,654::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:27,655::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:27,655::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:27,655::__init__::232::IOProcess::(_processLogs) (848) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:27,655::__init__::232::IOProcess::(_processLogs) (848) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:28,290::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:28,291::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1007::DEBUG::2014-10-15 07:09:28,292::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:31,300::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:31,300::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1008::DEBUG::2014-10-15 07:09:31,301::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:34,312::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:34,313::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1009::DEBUG::2014-10-15 07:09:34,314::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:37,324::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:37,325::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1010::DEBUG::2014-10-15 07:09:37,326::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:09:37,656::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:37,656::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:37,656::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:37,656::__init__::232::IOProcess::(_processLogs) (849) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:09:37,656::__init__::232::IOProcess::(_processLogs) (849) Queuing response >Thread-25::DEBUG::2014-10-15 07:09:37,657::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:09:37,663::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00040444 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:09:37,663::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:37,663::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:37,664::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:37,664::__init__::232::IOProcess::(_processLogs) (850) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:09:37,664::__init__::232::IOProcess::(_processLogs) (850) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:37,664::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:37,665::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:37,665::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:37,665::__init__::232::IOProcess::(_processLogs) (851) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:37,665::__init__::232::IOProcess::(_processLogs) (851) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:37,665::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:37,666::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:37,666::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:37,666::__init__::232::IOProcess::(_processLogs) (852) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:37,666::__init__::232::IOProcess::(_processLogs) (852) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:37,666::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:37,666::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:37,666::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:37,666::__init__::232::IOProcess::(_processLogs) (853) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:37,666::__init__::232::IOProcess::(_processLogs) (853) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:40,336::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:40,337::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1011::DEBUG::2014-10-15 07:09:40,338::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6407a7f-e0c2-4211-ac96-1fcd269e27c6`::moving from state init -> state preparing >Thread-1011::INFO::2014-10-15 07:09:40,338::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1011::INFO::2014-10-15 07:09:40,338::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00040444', 'lastCheck': '2.7', 'valid': True}} >Thread-1011::DEBUG::2014-10-15 07:09:40,338::task::1191::Storage.TaskManager.Task::(prepare) Task=`a6407a7f-e0c2-4211-ac96-1fcd269e27c6`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00040444', 'lastCheck': '2.7', 'valid': True}} >Thread-1011::DEBUG::2014-10-15 07:09:40,338::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6407a7f-e0c2-4211-ac96-1fcd269e27c6`::moving from state preparing -> state finished >Thread-1011::DEBUG::2014-10-15 07:09:40,339::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1011::DEBUG::2014-10-15 07:09:40,339::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1011::DEBUG::2014-10-15 07:09:40,339::task::993::Storage.TaskManager.Task::(_decref) Task=`a6407a7f-e0c2-4211-ac96-1fcd269e27c6`::ref 0 aborting False >Thread-1011::ERROR::2014-10-15 07:09:40,340::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1011::DEBUG::2014-10-15 07:09:40,340::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:40,351::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:40,352::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1012::DEBUG::2014-10-15 07:09:40,352::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:43,417::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:43,418::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1013::DEBUG::2014-10-15 07:09:43,419::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:46,429::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:46,430::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1014::DEBUG::2014-10-15 07:09:46,431::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:09:47,667::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:47,668::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:47,668::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:47,668::__init__::232::IOProcess::(_processLogs) (854) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:09:47,668::__init__::232::IOProcess::(_processLogs) (854) Queuing response >Thread-25::DEBUG::2014-10-15 07:09:47,668::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:09:47,674::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00040725 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:09:47,675::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:47,675::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:47,675::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:47,675::__init__::232::IOProcess::(_processLogs) (855) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:09:47,675::__init__::232::IOProcess::(_processLogs) (855) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:47,676::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:47,676::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:47,676::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:47,676::__init__::232::IOProcess::(_processLogs) (856) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:47,676::__init__::232::IOProcess::(_processLogs) (856) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:47,677::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:47,677::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:47,677::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:47,677::__init__::232::IOProcess::(_processLogs) (857) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:47,677::__init__::232::IOProcess::(_processLogs) (857) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:47,677::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:47,678::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:47,678::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:47,678::__init__::232::IOProcess::(_processLogs) (858) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:47,678::__init__::232::IOProcess::(_processLogs) (858) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:49,441::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:49,441::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1015::DEBUG::2014-10-15 07:09:49,442::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:52,449::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:52,450::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1016::DEBUG::2014-10-15 07:09:52,451::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:55,458::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:55,459::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1017::DEBUG::2014-10-15 07:09:55,460::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b2dc077-b0aa-46be-9c21-e7189710dde5`::moving from state init -> state preparing >Thread-1017::INFO::2014-10-15 07:09:55,461::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1017::INFO::2014-10-15 07:09:55,461::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00040725', 'lastCheck': '7.8', 'valid': True}} >Thread-1017::DEBUG::2014-10-15 07:09:55,461::task::1191::Storage.TaskManager.Task::(prepare) Task=`7b2dc077-b0aa-46be-9c21-e7189710dde5`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00040725', 'lastCheck': '7.8', 'valid': True}} >Thread-1017::DEBUG::2014-10-15 07:09:55,461::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b2dc077-b0aa-46be-9c21-e7189710dde5`::moving from state preparing -> state finished >Thread-1017::DEBUG::2014-10-15 07:09:55,461::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1017::DEBUG::2014-10-15 07:09:55,461::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1017::DEBUG::2014-10-15 07:09:55,461::task::993::Storage.TaskManager.Task::(_decref) Task=`7b2dc077-b0aa-46be-9c21-e7189710dde5`::ref 0 aborting False >Thread-1017::ERROR::2014-10-15 07:09:55,462::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1017::DEBUG::2014-10-15 07:09:55,463::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:55,473::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:55,473::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1018::DEBUG::2014-10-15 07:09:55,474::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:09:57,679::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:57,679::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:57,679::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:57,679::__init__::232::IOProcess::(_processLogs) (859) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:09:57,679::__init__::232::IOProcess::(_processLogs) (859) Queuing response >Thread-25::DEBUG::2014-10-15 07:09:57,680::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:09:57,686::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000379428 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:09:57,687::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:57,687::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:57,687::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:57,687::__init__::232::IOProcess::(_processLogs) (860) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:09:57,687::__init__::232::IOProcess::(_processLogs) (860) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:57,688::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:57,688::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:57,688::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:57,688::__init__::232::IOProcess::(_processLogs) (861) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:57,688::__init__::232::IOProcess::(_processLogs) (861) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:57,688::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:57,688::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:57,689::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:57,689::__init__::232::IOProcess::(_processLogs) (862) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:57,689::__init__::232::IOProcess::(_processLogs) (862) Queuing response >Thread-16::DEBUG::2014-10-15 07:09:57,689::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:09:57,689::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:09:57,689::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:09:57,689::__init__::232::IOProcess::(_processLogs) (863) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:09:57,689::__init__::232::IOProcess::(_processLogs) (863) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:09:58,570::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:09:58,571::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1019::DEBUG::2014-10-15 07:09:58,572::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:01,582::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:01,584::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1020::DEBUG::2014-10-15 07:10:01,586::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:04,596::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:04,597::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1021::DEBUG::2014-10-15 07:10:04,598::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:07,609::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:07,610::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1022::DEBUG::2014-10-15 07:10:07,611::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:10:07,690::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:07,691::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:07,691::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:07,691::__init__::232::IOProcess::(_processLogs) (864) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:10:07,691::__init__::232::IOProcess::(_processLogs) (864) Queuing response >Thread-25::DEBUG::2014-10-15 07:10:07,692::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:10:07,699::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00045204 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:10:07,700::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:07,700::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:07,700::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:07,700::__init__::232::IOProcess::(_processLogs) (865) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:10:07,700::__init__::232::IOProcess::(_processLogs) (865) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:07,701::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:07,701::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:07,701::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:07,701::__init__::232::IOProcess::(_processLogs) (866) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:07,701::__init__::232::IOProcess::(_processLogs) (866) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:07,701::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:07,702::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:07,702::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:07,702::__init__::232::IOProcess::(_processLogs) (867) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:07,702::__init__::232::IOProcess::(_processLogs) (867) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:07,702::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:07,702::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:07,702::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:07,702::__init__::232::IOProcess::(_processLogs) (868) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:07,703::__init__::232::IOProcess::(_processLogs) (868) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:10,621::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:10,622::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1023::DEBUG::2014-10-15 07:10:10,623::task::595::Storage.TaskManager.Task::(_updateState) Task=`a7087258-3c40-4076-a426-c6e187020226`::moving from state init -> state preparing >Thread-1023::INFO::2014-10-15 07:10:10,623::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1023::INFO::2014-10-15 07:10:10,624::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00045204', 'lastCheck': '2.9', 'valid': True}} >Thread-1023::DEBUG::2014-10-15 07:10:10,624::task::1191::Storage.TaskManager.Task::(prepare) Task=`a7087258-3c40-4076-a426-c6e187020226`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00045204', 'lastCheck': '2.9', 'valid': True}} >Thread-1023::DEBUG::2014-10-15 07:10:10,624::task::595::Storage.TaskManager.Task::(_updateState) Task=`a7087258-3c40-4076-a426-c6e187020226`::moving from state preparing -> state finished >Thread-1023::DEBUG::2014-10-15 07:10:10,624::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1023::DEBUG::2014-10-15 07:10:10,624::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1023::DEBUG::2014-10-15 07:10:10,624::task::993::Storage.TaskManager.Task::(_decref) Task=`a7087258-3c40-4076-a426-c6e187020226`::ref 0 aborting False >Thread-1023::ERROR::2014-10-15 07:10:10,625::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1023::DEBUG::2014-10-15 07:10:10,625::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:10,634::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:10,634::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1024::DEBUG::2014-10-15 07:10:10,635::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:13,720::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:13,721::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1025::DEBUG::2014-10-15 07:10:13,722::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:16,732::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:16,733::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1026::DEBUG::2014-10-15 07:10:16,734::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:10:17,704::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:17,704::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:17,704::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:17,704::__init__::232::IOProcess::(_processLogs) (869) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:10:17,704::__init__::232::IOProcess::(_processLogs) (869) Queuing response >Thread-25::DEBUG::2014-10-15 07:10:17,704::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:10:17,711::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000422827 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:10:17,712::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:17,712::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:17,712::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:17,712::__init__::232::IOProcess::(_processLogs) (870) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:10:17,712::__init__::232::IOProcess::(_processLogs) (870) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:17,713::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:17,713::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:17,713::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:17,713::__init__::232::IOProcess::(_processLogs) (871) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:17,713::__init__::232::IOProcess::(_processLogs) (871) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:17,713::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:17,713::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:17,714::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:17,714::__init__::232::IOProcess::(_processLogs) (872) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:17,714::__init__::232::IOProcess::(_processLogs) (872) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:17,714::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:17,714::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:17,714::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:17,714::__init__::232::IOProcess::(_processLogs) (873) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:17,714::__init__::232::IOProcess::(_processLogs) (873) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:19,746::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:19,746::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1027::DEBUG::2014-10-15 07:10:19,747::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:22,759::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:22,760::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1028::DEBUG::2014-10-15 07:10:22,761::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:25,771::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:25,772::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1029::DEBUG::2014-10-15 07:10:25,773::task::595::Storage.TaskManager.Task::(_updateState) Task=`202b6423-3b8d-43b4-b5f6-d4cfdb0dac50`::moving from state init -> state preparing >Thread-1029::INFO::2014-10-15 07:10:25,773::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1029::INFO::2014-10-15 07:10:25,773::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422827', 'lastCheck': '8.1', 'valid': True}} >Thread-1029::DEBUG::2014-10-15 07:10:25,774::task::1191::Storage.TaskManager.Task::(prepare) Task=`202b6423-3b8d-43b4-b5f6-d4cfdb0dac50`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422827', 'lastCheck': '8.1', 'valid': True}} >Thread-1029::DEBUG::2014-10-15 07:10:25,774::task::595::Storage.TaskManager.Task::(_updateState) Task=`202b6423-3b8d-43b4-b5f6-d4cfdb0dac50`::moving from state preparing -> state finished >Thread-1029::DEBUG::2014-10-15 07:10:25,774::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1029::DEBUG::2014-10-15 07:10:25,774::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1029::DEBUG::2014-10-15 07:10:25,774::task::993::Storage.TaskManager.Task::(_decref) Task=`202b6423-3b8d-43b4-b5f6-d4cfdb0dac50`::ref 0 aborting False >Thread-1029::ERROR::2014-10-15 07:10:25,775::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1029::DEBUG::2014-10-15 07:10:25,775::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:25,785::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:25,786::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1030::DEBUG::2014-10-15 07:10:25,787::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:10:27,715::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:27,716::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:27,716::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:27,716::__init__::232::IOProcess::(_processLogs) (874) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:10:27,716::__init__::232::IOProcess::(_processLogs) (874) Queuing response >Thread-25::DEBUG::2014-10-15 07:10:27,716::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:10:27,722::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000358407 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:10:27,723::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:27,723::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:27,723::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:27,723::__init__::232::IOProcess::(_processLogs) (875) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:10:27,723::__init__::232::IOProcess::(_processLogs) (875) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:27,724::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:27,724::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:27,724::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:27,724::__init__::232::IOProcess::(_processLogs) (876) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:27,724::__init__::232::IOProcess::(_processLogs) (876) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:27,725::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:27,725::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:27,725::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:27,725::__init__::232::IOProcess::(_processLogs) (877) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:27,725::__init__::232::IOProcess::(_processLogs) (877) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:27,725::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:27,725::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:27,726::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:27,726::__init__::232::IOProcess::(_processLogs) (878) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:27,726::__init__::232::IOProcess::(_processLogs) (878) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:28,842::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:28,843::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1031::DEBUG::2014-10-15 07:10:28,844::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:31,850::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:31,851::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1032::DEBUG::2014-10-15 07:10:31,854::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:34,863::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:34,863::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1033::DEBUG::2014-10-15 07:10:34,864::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:10:37,727::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:37,727::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:37,727::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:37,727::__init__::232::IOProcess::(_processLogs) (879) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:10:37,727::__init__::232::IOProcess::(_processLogs) (879) Queuing response >Thread-25::DEBUG::2014-10-15 07:10:37,728::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:10:37,734::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000421196 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:10:37,734::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:37,735::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:37,735::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:37,735::__init__::232::IOProcess::(_processLogs) (880) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:10:37,735::__init__::232::IOProcess::(_processLogs) (880) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:37,735::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:37,736::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:37,736::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:37,736::__init__::232::IOProcess::(_processLogs) (881) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:37,736::__init__::232::IOProcess::(_processLogs) (881) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:37,736::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:37,736::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:37,736::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:37,737::__init__::232::IOProcess::(_processLogs) (882) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:37,737::__init__::232::IOProcess::(_processLogs) (882) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:37,737::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:37,737::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:37,737::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:37,737::__init__::232::IOProcess::(_processLogs) (883) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:37,737::__init__::232::IOProcess::(_processLogs) (883) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:37,875::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:37,876::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1034::DEBUG::2014-10-15 07:10:37,877::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:40,887::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:40,888::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1035::DEBUG::2014-10-15 07:10:40,889::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4314ba3-8889-47ac-a9b8-5fda94fcf830`::moving from state init -> state preparing >Thread-1035::INFO::2014-10-15 07:10:40,889::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1035::INFO::2014-10-15 07:10:40,889::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421196', 'lastCheck': '3.2', 'valid': True}} >Thread-1035::DEBUG::2014-10-15 07:10:40,889::task::1191::Storage.TaskManager.Task::(prepare) Task=`d4314ba3-8889-47ac-a9b8-5fda94fcf830`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421196', 'lastCheck': '3.2', 'valid': True}} >Thread-1035::DEBUG::2014-10-15 07:10:40,890::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4314ba3-8889-47ac-a9b8-5fda94fcf830`::moving from state preparing -> state finished >Thread-1035::DEBUG::2014-10-15 07:10:40,890::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1035::DEBUG::2014-10-15 07:10:40,890::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1035::DEBUG::2014-10-15 07:10:40,890::task::993::Storage.TaskManager.Task::(_decref) Task=`d4314ba3-8889-47ac-a9b8-5fda94fcf830`::ref 0 aborting False >Thread-1035::ERROR::2014-10-15 07:10:40,891::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1035::DEBUG::2014-10-15 07:10:40,891::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:40,899::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:40,900::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1036::DEBUG::2014-10-15 07:10:40,900::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:43,966::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:43,967::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1037::DEBUG::2014-10-15 07:10:43,968::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:47,004::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:47,004::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1038::DEBUG::2014-10-15 07:10:47,005::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:10:47,738::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:47,738::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:47,739::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:47,739::__init__::232::IOProcess::(_processLogs) (884) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:10:47,739::__init__::232::IOProcess::(_processLogs) (884) Queuing response >Thread-25::DEBUG::2014-10-15 07:10:47,739::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:10:47,745::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000393314 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:10:47,746::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:47,746::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:47,746::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:47,747::__init__::232::IOProcess::(_processLogs) (885) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:10:47,747::__init__::232::IOProcess::(_processLogs) (885) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:47,747::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:47,747::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:47,748::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:47,748::__init__::232::IOProcess::(_processLogs) (886) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:47,748::__init__::232::IOProcess::(_processLogs) (886) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:47,748::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:47,748::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:47,748::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:47,748::__init__::232::IOProcess::(_processLogs) (887) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:47,749::__init__::232::IOProcess::(_processLogs) (887) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:47,749::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:47,749::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:47,749::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:47,749::__init__::232::IOProcess::(_processLogs) (888) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:47,749::__init__::232::IOProcess::(_processLogs) (888) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:50,016::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:50,017::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1039::DEBUG::2014-10-15 07:10:50,018::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:53,030::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:53,030::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1040::DEBUG::2014-10-15 07:10:53,031::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:56,043::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:56,043::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1041::DEBUG::2014-10-15 07:10:56,045::task::595::Storage.TaskManager.Task::(_updateState) Task=`646213c7-e497-45cc-8dbf-e4bbde3c90a8`::moving from state init -> state preparing >Thread-1041::INFO::2014-10-15 07:10:56,045::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1041::INFO::2014-10-15 07:10:56,045::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000393314', 'lastCheck': '8.3', 'valid': True}} >Thread-1041::DEBUG::2014-10-15 07:10:56,045::task::1191::Storage.TaskManager.Task::(prepare) Task=`646213c7-e497-45cc-8dbf-e4bbde3c90a8`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000393314', 'lastCheck': '8.3', 'valid': True}} >Thread-1041::DEBUG::2014-10-15 07:10:56,045::task::595::Storage.TaskManager.Task::(_updateState) Task=`646213c7-e497-45cc-8dbf-e4bbde3c90a8`::moving from state preparing -> state finished >Thread-1041::DEBUG::2014-10-15 07:10:56,045::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1041::DEBUG::2014-10-15 07:10:56,046::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1041::DEBUG::2014-10-15 07:10:56,046::task::993::Storage.TaskManager.Task::(_decref) Task=`646213c7-e497-45cc-8dbf-e4bbde3c90a8`::ref 0 aborting False >Thread-1041::ERROR::2014-10-15 07:10:56,046::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1041::DEBUG::2014-10-15 07:10:56,047::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:56,060::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:56,060::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1042::DEBUG::2014-10-15 07:10:56,061::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:10:57,750::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:57,751::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:57,751::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:57,751::__init__::232::IOProcess::(_processLogs) (889) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:10:57,751::__init__::232::IOProcess::(_processLogs) (889) Queuing response >Thread-25::DEBUG::2014-10-15 07:10:57,751::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:10:57,758::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00038094 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:10:57,758::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:57,758::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:57,759::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:57,759::__init__::232::IOProcess::(_processLogs) (890) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:10:57,759::__init__::232::IOProcess::(_processLogs) (890) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:57,759::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:57,759::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:57,760::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:57,760::__init__::232::IOProcess::(_processLogs) (891) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:57,760::__init__::232::IOProcess::(_processLogs) (891) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:57,760::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:57,760::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:57,760::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:57,760::__init__::232::IOProcess::(_processLogs) (892) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:57,760::__init__::232::IOProcess::(_processLogs) (892) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:57,761::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:57,761::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:57,761::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:57,761::__init__::232::IOProcess::(_processLogs) (893) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:57,761::__init__::232::IOProcess::(_processLogs) (893) Queuing response >Detector thread::DEBUG::2014-10-15 07:10:58,781::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.66.100.111:54929 >Detector thread::DEBUG::2014-10-15 07:10:58,782::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.66.100.111:54929 >Detector thread::DEBUG::2014-10-15 07:10:58,782::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.66.100.111:54929 >Detector thread::DEBUG::2014-10-15 07:10:58,782::BindingXMLRPC::1172::XmlDetector::(handleSocket) xml over http detected from ('10.66.100.111', 54929) >Thread-1043::DEBUG::2014-10-15 07:10:58,783::BindingXMLRPC::1132::vds::(wrapper) client [10.66.100.111]::call vmGetStats with ('c255610f-42c2-426d-9270-e4497f4a0f37',) {} >Thread-1043::DEBUG::2014-10-15 07:10:58,783::BindingXMLRPC::1139::vds::(wrapper) return vmGetStats with {'status': {'message': 'Virtual machine does not exist', 'code': 1}} >Detector thread::DEBUG::2014-10-15 07:10:58,809::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.66.100.111:54930 >Detector thread::DEBUG::2014-10-15 07:10:58,810::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.66.100.111:54930 >Detector thread::DEBUG::2014-10-15 07:10:58,810::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.66.100.111:54930 >Detector thread::DEBUG::2014-10-15 07:10:58,810::BindingXMLRPC::1172::XmlDetector::(handleSocket) xml over http detected from ('10.66.100.111', 54930) >Thread-1044::DEBUG::2014-10-15 07:10:58,818::BindingXMLRPC::1132::vds::(wrapper) client [10.66.100.111]::call vmMigrationCreate with ({'guestFQDN': '', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.5.0', 'afterMigrationStatus': '', 'spiceSecureChannels': 'smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard', 'vmId': 'c255610f-42c2-426d-9270-e4497f4a0f37', 'memGuaranteedSize': 1024, 'transparentHugePages': 'true', 'displaySecurePort': '5900', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'SandyBridge', 'smp': '1', 'migrationDest': 'libvirt', 'custom': {'device_f47764b9-9bdd-497e-aa23-ac4b42744926device_8637879d-080d-4b8e-b157-43a915564601device_72fc718b-c615-400d-981a-ad80b846f515device_d45c1a74-983c-4785-b321-73e4f4191870': 'VmDevice {vmId=c255610f-42c2-426d-9270-e4497f4a0f37, deviceId=d45c1a74-983c-4785-b321-73e4f4191870, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=3}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}', 'device_f47764b9-9bdd-497e-aa23-ac4b42744926device_8637879d-080d-4b8e-b157-43a915564601': 'VmDevice {vmId=c255610f-42c2-426d-9270-e4497f4a0f37, deviceId=8637879d-080d-4b8e-b157-43a915564601, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=1}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}', 'device_f47764b9-9bdd-497e-aa23-ac4b42744926device_8637879d-080d-4b8e-b157-43a915564601device_72fc718b-c615-400d-981a-ad80b846f515': 'VmDevice {vmId=c255610f-42c2-426d-9270-e4497f4a0f37, deviceId=72fc718b-c615-400d-981a-ad80b846f515, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=2}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}', 'device_f47764b9-9bdd-497e-aa23-ac4b42744926': 'VmDevice {vmId=c255610f-42c2-426d-9270-e4497f4a0f37, deviceId=f47764b9-9bdd-497e-aa23-ac4b42744926, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}'}, 'vmType': 'kvm', '_srcDomXML': "<domain type='kvm' id='2'>\n <name>rhevh-guest-rhel7.0</name>\n <uuid>c255610f-42c2-426d-9270-e4497f4a0f37</uuid>\n <memory unit='KiB'>1048576</memory>\n <currentMemory unit='KiB'>1048576</currentMemory>\n <vcpu placement='static' current='1'>16</vcpu>\n <cputune>\n <shares>1020</shares>\n </cputune>\n <resource>\n <partition>/machine</partition>\n </resource>\n <sysinfo type='smbios'>\n <system>\n <entry name='manufacturer'>Red Hat</entry>\n <entry name='product'>RHEV Hypervisor</entry>\n <entry name='version'>7.0-20141006.0.el7ev</entry>\n <entry name='serial'>A63BD200-CD0E-11E1-9290-A0B3CCF7E533</entry>\n <entry name='uuid'>c255610f-42c2-426d-9270-e4497f4a0f37</entry>\n </system>\n </sysinfo>\n <os>\n <type arch='x86_64' machine='rhel6.5.0'>hvm</type>\n <smbios mode='sysinfo'/>\n </os>\n <features>\n <acpi/>\n </features>\n <cpu mode='custom' match='exact'>\n <model fallback='allow'>SandyBridge</model>\n <topology sockets='16' cores='1' threads='1'/>\n </cpu>\n <clock offset='variable' adjustment='0' basis='utc'>\n <timer name='rtc' tickpolicy='catchup'/>\n <timer name='pit' tickpolicy='delay'/>\n <timer name='hpet' present='no'/>\n </clock>\n <on_poweroff>destroy</on_poweroff>\n <on_reboot>restart</on_reboot>\n <on_crash>destroy</on_crash>\n <devices>\n <emulator>/usr/libexec/qemu-kvm</emulator>\n <disk type='file' device='cdrom'>\n <driver name='qemu' type='raw'/>\n <source startupPolicy='optional'/>\n <target dev='hdc' bus='ide'/>\n <readonly/>\n <serial></serial>\n <alias name='ide0-1-0'/>\n <address type='drive' controller='0' bus='1' target='0' unit='0'/>\n </disk>\n <disk type='file' device='disk' snapshot='no'>\n <driver name='qemu' type='raw' cache='none' error_policy='stop' io='threads'/>\n <source file='/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb'>\n <seclabel model='selinux' labelskip='yes'/>\n </source>\n <target dev='vda' bus='virtio'/>\n <serial>a5611808-d282-45f9-9771-c3797b521610</serial>\n <boot order='2'/>\n <alias name='virtio-disk0'/>\n <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>\n </disk>\n <controller type='scsi' index='0' model='virtio-scsi'>\n <alias name='scsi0'/>\n <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>\n </controller>\n <controller type='virtio-serial' index='0' ports='16'>\n <alias name='virtio-serial0'/>\n <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>\n </controller>\n <controller type='ide' index='0'>\n <alias name='ide0'/>\n <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>\n </controller>\n <controller type='usb' index='0'>\n <alias name='usb0'/>\n <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>\n </controller>\n <controller type='pci' index='0' model='pci-root'>\n <alias name='pci.0'/>\n </controller>\n <interface type='bridge'>\n <mac address='00:1a:4a:ad:8d:00'/>\n <source bridge='rhevm'/>\n <target dev='vnet0'/>\n <model type='virtio'/>\n <filterref filter='vdsm-no-mac-spoofing'/>\n <link state='up'/>\n <bandwidth>\n </bandwidth>\n <boot order='1'/>\n <alias name='net0'/>\n <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>\n </interface>\n <channel type='unix'>\n <source mode='bind' path='/var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm'/>\n <target type='virtio' name='com.redhat.rhevm.vdsm'/>\n <alias name='channel0'/>\n <address type='virtio-serial' controller='0' bus='0' port='1'/>\n </channel>\n <channel type='unix'>\n <source mode='bind' path='/var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.org.qemu.guest_agent.0'/>\n <target type='virtio' name='org.qemu.guest_agent.0'/>\n <alias name='channel1'/>\n <address type='virtio-serial' controller='0' bus='0' port='2'/>\n </channel>\n <channel type='spicevmc'>\n <target type='virtio' name='com.redhat.spice.0'/>\n <alias name='channel2'/>\n <address type='virtio-serial' controller='0' bus='0' port='3'/>\n </channel>\n <input type='mouse' bus='ps2'/>\n <graphics type='spice' tlsPort='5900' autoport='yes' keymap='en-us' passwdValidTo='1970-01-01T00:00:01'>\n <listen type='network' address='10.66.100.111' network='vdsm-rhevm'/>\n <channel name='main' mode='secure'/>\n <channel name='display' mode='secure'/>\n <channel name='inputs' mode='secure'/>\n <channel name='cursor' mode='secure'/>\n <channel name='playback' mode='secure'/>\n <channel name='record' mode='secure'/>\n <channel name='smartcard' mode='secure'/>\n <channel name='usbredir' mode='secure'/>\n </graphics>\n <video>\n <model type='qxl' ram='65536' vram='32768' heads='1'/>\n <alias name='video0'/>\n <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>\n </video>\n <memballoon model='virtio'>\n <alias name='balloon0'/>\n <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>\n </memballoon>\n </devices>\n <seclabel type='dynamic' model='selinux' relabel='yes'>\n <label>system_u:system_r:svirt_t:s0:c367,c484</label>\n <imagelabel>system_u:object_r:svirt_image_t:s0:c367,c484</imagelabel>\n </seclabel>\n</domain>\n", 'memSize': 1024, 'smpCoresPerSocket': '1', 'vmName': 'rhevh-guest-rhel7.0', 'nice': '0', 'username': 'Unknown', 'bootMenuEnable': 'false', 'copyPasteEnable': 'true', 'displayIp': '10.66.100.111', 'keyboardLayout': 'en-us', 'displayPort': '-1', 'smartcardEnable': 'false', 'clientIp': '', 'fileTransferEnable': 'true', 'nicModel': 'rtl8139,pv', 'elapsedTimeOffset': 6057.54408788681, 'kvmEnable': 'true', 'displayNetwork': 'rhevm', 'devices': [{'target': 1048576, 'alias': 'balloon0', 'specParams': {'model': 'virtio'}, 'deviceType': 'balloon', 'deviceId': '85359974-17aa-43a3-a43a-b66ead768ca8', 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'memballoon', 'type': 'balloon'}, {'device': 'unix', 'alias': 'channel0', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '1'}, 'deviceType': 'channel', 'type': 'channel'}, {'device': 'unix', 'alias': 'channel1', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '2'}, 'deviceType': 'channel', 'type': 'channel'}, {'device': 'spicevmc', 'alias': 'channel2', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '3'}, 'deviceType': 'channel', 'type': 'channel'}, {'index': '0', 'alias': 'scsi0', 'specParams': {}, 'deviceType': 'controller', 'deviceId': 'f6a83fc4-d8ba-4522-851c-112c295ac642', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}, {'alias': 'virtio-serial0', 'specParams': {}, 'deviceType': 'controller', 'deviceId': '5430e069-f469-4ff7-a9c4-e234e970951d', 'address': {'slot': '0x04', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'virtio-serial', 'type': 'controller'}, {'device': 'usb', 'alias': 'usb0', 'address': {'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x2'}, 'deviceType': 'controller', 'type': 'controller'}, {'device': 'ide', 'alias': 'ide0', 'address': {'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x1'}, 'deviceType': 'controller', 'type': 'controller'}, {'alias': 'video0', 'specParams': {'vram': '32768', 'heads': '1'}, 'deviceType': 'video', 'deviceId': '7dc39394-a63a-4b22-b799-7d36c954396d', 'address': {'slot': '0x02', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'qxl', 'type': 'video'}, {'device': 'spice', 'specParams': {'copyPasteEnable': 'true', 'displayNetwork': 'rhevm', 'keyMap': 'en-us', 'displayIp': '10.66.100.111', 'spiceSecureChannels': 'smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard'}, 'type': 'graphics', 'deviceType': 'graphics', 'tlsPort': '5900'}, {'nicModel': 'pv', 'macAddr': '00:1a:4a:ad:8d:00', 'linkActive': True, 'network': 'rhevm', 'alias': 'net0', 'bootOrder': '1', 'filter': 'vdsm-no-mac-spoofing', 'specParams': {'inbound': {}, 'outbound': {}}, 'deviceType': 'interface', 'deviceId': 'd438dab9-542d-4077-8515-f30b69256612', 'address': {'slot': '0x07', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface', 'name': 'vnet0'}, {'index': '2', 'iface': 'ide', 'name': 'hdc', 'alias': 'ide0-1-0', 'specParams': {'path': ''}, 'readonly': 'True', 'deviceType': 'disk', 'deviceId': 'f42525d0-86e2-43e6-a68e-0565b55dd504', 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'cdrom', 'shared': 'false', 'path': '', 'type': 'disk'}, {'address': {'slot': '0x05', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'volumeInfo': {'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'volumeID': '486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': '/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610', 'path': '/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb'}, 'index': 0, 'iface': 'virtio', 'apparentsize': '10737418240', 'alias': 'virtio-disk0', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610', 'readonly': 'False', 'deviceType': 'disk', 'shared': 'false', 'truesize': '4116598784', 'type': 'disk', 'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'reqsize': '0', 'format': 'raw', 'deviceId': 'a5611808-d282-45f9-9771-c3797b521610', 'poolID': '00000002-0002-0002-0002-0000000003e1', 'device': 'disk', 'path': '/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'propagateErrors': 'off', 'optional': 'false', 'name': 'vda', 'bootOrder': '2', 'volumeID': '486f9fc4-3108-49db-ae60-4dc85544e8eb', 'specParams': {}, 'volumeChain': [{'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'volumeID': '486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': '/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610', 'path': '/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb'}]}], 'status': 'Up', 'timeOffset': '0', 'maxVCpus': '16', 'guestIPs': '', 'display': 'qxl'},) {} >Thread-1044::DEBUG::2014-10-15 07:10:58,818::API::545::vds::(migrationCreate) Migration create >Thread-1044::INFO::2014-10-15 07:10:58,818::clientIF::390::vds::(createVm) vmContainerLock acquired by vm c255610f-42c2-426d-9270-e4497f4a0f37 >Thread-1044::DEBUG::2014-10-15 07:10:58,823::clientIF::403::vds::(createVm) Total desktops after creation of c255610f-42c2-426d-9270-e4497f4a0f37 is 1 >Thread-1045::DEBUG::2014-10-15 07:10:58,823::vm::2259::vm.Vm::(_startUnderlyingVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Start >Thread-1045::DEBUG::2014-10-15 07:10:58,824::vm::2263::vm.Vm::(_startUnderlyingVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::_ongoingCreations acquired >Thread-1044::DEBUG::2014-10-15 07:10:58,824::vm::5548::vm.Vm::(waitForMigrationDestinationPrepare) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::migration destination: waiting for VM creation >Thread-1045::INFO::2014-10-15 07:10:58,825::vm::3299::vm.Vm::(_run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::VM wrapper has started >Thread-1044::DEBUG::2014-10-15 07:10:58,825::vm::5553::vm.Vm::(waitForMigrationDestinationPrepare) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::migration destination: waiting 36s for path preparation >Thread-1045::WARNING::2014-10-15 07:10:58,826::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': 'unix', 'alias': 'channel0', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '1'}, 'deviceType': 'channel', 'type': 'channel'}' found >Thread-1045::WARNING::2014-10-15 07:10:58,826::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': 'unix', 'alias': 'channel1', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '2'}, 'deviceType': 'channel', 'type': 'channel'}' found >Thread-1045::WARNING::2014-10-15 07:10:58,826::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': 'spicevmc', 'alias': 'channel2', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '3'}, 'deviceType': 'channel', 'type': 'channel'}' found >Thread-1045::DEBUG::2014-10-15 07:10:58,826::task::595::Storage.TaskManager.Task::(_updateState) Task=`b1d166aa-7d85-42a7-8378-8481b6922e8b`::moving from state init -> state preparing >Thread-1045::INFO::2014-10-15 07:10:58,827::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', volUUID='486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:10:58,827::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:58,827::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:58,827::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:58,827::__init__::232::IOProcess::(_processLogs) (894) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:10:58,828::__init__::232::IOProcess::(_processLogs) (894) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:58,828::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:58,828::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:58,828::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:58,829::__init__::232::IOProcess::(_processLogs) (895) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:10:58,829::__init__::232::IOProcess::(_processLogs) (895) Queuing response >Thread-1045::INFO::2014-10-15 07:10:58,829::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1045::DEBUG::2014-10-15 07:10:58,829::task::1191::Storage.TaskManager.Task::(prepare) Task=`b1d166aa-7d85-42a7-8378-8481b6922e8b`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1045::DEBUG::2014-10-15 07:10:58,829::task::595::Storage.TaskManager.Task::(_updateState) Task=`b1d166aa-7d85-42a7-8378-8481b6922e8b`::moving from state preparing -> state finished >Thread-1045::DEBUG::2014-10-15 07:10:58,829::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1045::DEBUG::2014-10-15 07:10:58,829::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1045::DEBUG::2014-10-15 07:10:58,829::task::993::Storage.TaskManager.Task::(_decref) Task=`b1d166aa-7d85-42a7-8378-8481b6922e8b`::ref 0 aborting False >Thread-1045::INFO::2014-10-15 07:10:58,829::clientIF::331::vds::(prepareVolumePath) prepared volume path: >Thread-1045::DEBUG::2014-10-15 07:10:58,830::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed0058e0-c3ef-4aa8-8fc2-86231108373d`::moving from state init -> state preparing >Thread-1045::INFO::2014-10-15 07:10:58,830::logUtils::44::dispatcher::(wrapper) Run and protect: prepareImage(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', leafUUID='486f9fc4-3108-49db-ae60-4dc85544e8eb') >Thread-1045::DEBUG::2014-10-15 07:10:58,830::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4`ReqID=`8d68f87e-b95f-423d-8b2a-c68bab66f16f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '3210' at 'prepareImage' >Thread-1045::DEBUG::2014-10-15 07:10:58,830::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' for lock type 'shared' >Thread-1045::DEBUG::2014-10-15 07:10:58,830::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' is free. Now locking as 'shared' (1 active user) >Thread-1045::DEBUG::2014-10-15 07:10:58,830::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4`ReqID=`8d68f87e-b95f-423d-8b2a-c68bab66f16f`::Granted request >Thread-1045::DEBUG::2014-10-15 07:10:58,831::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ed0058e0-c3ef-4aa8-8fc2-86231108373d`::_resourcesAcquired: Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 (shared) >Thread-1045::DEBUG::2014-10-15 07:10:58,831::task::993::Storage.TaskManager.Task::(_decref) Task=`ed0058e0-c3ef-4aa8-8fc2-86231108373d`::ref 1 aborting False >Thread-16::DEBUG::2014-10-15 07:10:58,831::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:58,831::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:58,831::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:58,831::__init__::232::IOProcess::(_processLogs) (896) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:10:58,833::__init__::232::IOProcess::(_processLogs) (896) Queuing response >Thread-1045::DEBUG::2014-10-15 07:10:58,834::fileSD::499::Storage.StorageDomain::(activateVolumes) Fixing permissions on /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb >Thread-16::DEBUG::2014-10-15 07:10:58,834::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:58,834::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:58,834::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:58,834::__init__::232::IOProcess::(_processLogs) (897) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:10:58,834::__init__::232::IOProcess::(_processLogs) (897) Queuing response >Thread-1045::DEBUG::2014-10-15 07:10:58,834::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /var/run/vdsm/storage/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-1045::WARNING::2014-10-15 07:10:58,835::fileUtils::149::Storage.fileUtils::(createdir) Dir /var/run/vdsm/storage/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 already exists >Thread-1045::DEBUG::2014-10-15 07:10:58,835::fileSD::474::Storage.StorageDomain::(createImageLinks) Creating symlink from /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610 to /var/run/vdsm/storage/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/a5611808-d282-45f9-9771-c3797b521610 >Thread-1045::DEBUG::2014-10-15 07:10:58,835::fileSD::479::Storage.StorageDomain::(createImageLinks) img run dir already exists: /var/run/vdsm/storage/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/a5611808-d282-45f9-9771-c3797b521610 >Thread-16::DEBUG::2014-10-15 07:10:58,835::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:58,835::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:58,835::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:58,835::__init__::232::IOProcess::(_processLogs) (898) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:10:58,836::__init__::232::IOProcess::(_processLogs) (898) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:58,836::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:58,836::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:58,836::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:58,836::__init__::232::IOProcess::(_processLogs) (899) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:58,836::__init__::232::IOProcess::(_processLogs) (899) Queuing response >Thread-1045::DEBUG::2014-10-15 07:10:58,836::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for 486f9fc4-3108-49db-ae60-4dc85544e8eb >Thread-16::DEBUG::2014-10-15 07:10:58,837::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:58,837::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:58,837::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:58,837::__init__::232::IOProcess::(_processLogs) (900) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:58,837::__init__::232::IOProcess::(_processLogs) (900) Queuing response >Thread-16::DEBUG::2014-10-15 07:10:58,837::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:10:58,837::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:10:58,838::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:10:58,838::__init__::232::IOProcess::(_processLogs) (901) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:10:58,838::__init__::232::IOProcess::(_processLogs) (901) Queuing response >Thread-1045::INFO::2014-10-15 07:10:58,838::logUtils::47::dispatcher::(wrapper) Run and protect: prepareImage, Return response: {'info': {'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610'}, 'path': u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'imgVolumesInfo': [{'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610'}]} >Thread-1045::DEBUG::2014-10-15 07:10:58,838::task::1191::Storage.TaskManager.Task::(prepare) Task=`ed0058e0-c3ef-4aa8-8fc2-86231108373d`::finished: {'info': {'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610'}, 'path': u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'imgVolumesInfo': [{'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610'}]} >Thread-1045::DEBUG::2014-10-15 07:10:58,838::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed0058e0-c3ef-4aa8-8fc2-86231108373d`::moving from state preparing -> state finished >Thread-1045::DEBUG::2014-10-15 07:10:58,838::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': < ResourceRef 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', isValid: 'True' obj: 'None'>} >Thread-1045::DEBUG::2014-10-15 07:10:58,838::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1045::DEBUG::2014-10-15 07:10:58,839::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' >Thread-1045::DEBUG::2014-10-15 07:10:58,839::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' (0 active users) >Thread-1045::DEBUG::2014-10-15 07:10:58,839::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' is free, finding out if anyone is waiting for it. >Thread-1045::DEBUG::2014-10-15 07:10:58,839::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', Clearing records. >Thread-1045::DEBUG::2014-10-15 07:10:58,839::task::993::Storage.TaskManager.Task::(_decref) Task=`ed0058e0-c3ef-4aa8-8fc2-86231108373d`::ref 0 aborting False >Thread-1045::INFO::2014-10-15 07:10:58,839::clientIF::331::vds::(prepareVolumePath) prepared volume path: /rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb >Thread-1045::DEBUG::2014-10-15 07:10:58,842::vm::1067::vm.Vm::(__init__) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Ignoring param (target, 1048576) in BalloonDevice >Thread-1045::DEBUG::2014-10-15 07:10:58,843::vm::2289::vm.Vm::(_startUnderlyingVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::_ongoingCreations released >Thread-1044::ERROR::2014-10-15 07:10:58,843::vm::5528::vm.Vm::(_updateDevicesDomxmlCache) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Alias not found for device type graphics during migration at destination host >Thread-1045::DEBUG::2014-10-15 07:10:58,843::vm::4021::vm.Vm::(_waitForUnderlyingMigration) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Waiting 21600 seconds for end of migration >Thread-1044::DEBUG::2014-10-15 07:10:58,850::utils::738::root::(execCmd) /usr/libexec/vdsm/hooks/before_vm_migrate_destination/50_vhostmd (cwd None) >Thread-1044::DEBUG::2014-10-15 07:10:58,924::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-1044::INFO::2014-10-15 07:10:58,925::hooks::100::root::(_runHooksDir) >Thread-1044::DEBUG::2014-10-15 07:10:58,925::API::558::vds::(migrationCreate) Destination VM creation succeeded >Thread-1044::DEBUG::2014-10-15 07:10:58,925::BindingXMLRPC::1139::vds::(wrapper) return vmMigrationCreate with {'status': {'message': 'Done', 'code': 0}, 'migrationPort': 0, 'params': {'guestFQDN': '', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.5.0', 'afterMigrationStatus': '', 'spiceSecureChannels': 'smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard', 'vmId': 'c255610f-42c2-426d-9270-e4497f4a0f37', 'memGuaranteedSize': 1024, 'transparentHugePages': 'true', 'displaySecurePort': '-1', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'SandyBridge', 'custom': {'device_f47764b9-9bdd-497e-aa23-ac4b42744926device_8637879d-080d-4b8e-b157-43a915564601device_72fc718b-c615-400d-981a-ad80b846f515device_d45c1a74-983c-4785-b321-73e4f4191870': 'VmDevice {vmId=c255610f-42c2-426d-9270-e4497f4a0f37, deviceId=d45c1a74-983c-4785-b321-73e4f4191870, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=3}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}', 'device_f47764b9-9bdd-497e-aa23-ac4b42744926device_8637879d-080d-4b8e-b157-43a915564601': 'VmDevice {vmId=c255610f-42c2-426d-9270-e4497f4a0f37, deviceId=8637879d-080d-4b8e-b157-43a915564601, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=1}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}', 'device_f47764b9-9bdd-497e-aa23-ac4b42744926device_8637879d-080d-4b8e-b157-43a915564601device_72fc718b-c615-400d-981a-ad80b846f515': 'VmDevice {vmId=c255610f-42c2-426d-9270-e4497f4a0f37, deviceId=72fc718b-c615-400d-981a-ad80b846f515, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=2}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}', 'device_f47764b9-9bdd-497e-aa23-ac4b42744926': 'VmDevice {vmId=c255610f-42c2-426d-9270-e4497f4a0f37, deviceId=f47764b9-9bdd-497e-aa23-ac4b42744926, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}'}, 'migrationDest': 'libvirt', 'smp': '1', 'vmType': 'kvm', 'memSize': 1024, 'smpCoresPerSocket': '1', 'vmName': 'rhevh-guest-rhel7.0', 'nice': '0', 'username': 'Unknown', 'bootMenuEnable': 'false', 'pid': '0', 'copyPasteEnable': 'true', 'displayIp': '10.66.100.109', 'displayPort': '-1', 'smartcardEnable': 'false', 'clientIp': '', 'fileTransferEnable': 'true', 'nicModel': 'rtl8139,pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'displayNetwork': 'rhevm', 'devices': [{'target': 1048576, 'alias': 'balloon0', 'specParams': {'model': 'virtio'}, 'deviceType': 'balloon', 'deviceId': '85359974-17aa-43a3-a43a-b66ead768ca8', 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'memballoon', 'type': 'balloon'}, {'device': 'unix', 'alias': 'channel0', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '1'}, 'deviceType': 'channel', 'type': 'channel'}, {'device': 'unix', 'alias': 'channel1', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '2'}, 'deviceType': 'channel', 'type': 'channel'}, {'device': 'spicevmc', 'alias': 'channel2', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '3'}, 'deviceType': 'channel', 'type': 'channel'}, {'index': '0', 'alias': 'scsi0', 'specParams': {}, 'deviceType': 'controller', 'deviceId': 'f6a83fc4-d8ba-4522-851c-112c295ac642', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}, {'alias': 'virtio-serial0', 'specParams': {}, 'deviceType': 'controller', 'deviceId': '5430e069-f469-4ff7-a9c4-e234e970951d', 'address': {'slot': '0x04', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'virtio-serial', 'type': 'controller'}, {'device': 'usb', 'alias': 'usb0', 'address': {'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x2'}, 'deviceType': 'controller', 'type': 'controller'}, {'device': 'ide', 'alias': 'ide0', 'address': {'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x1'}, 'deviceType': 'controller', 'type': 'controller'}, {'alias': 'video0', 'specParams': {'vram': '32768', 'heads': '1'}, 'deviceType': 'video', 'deviceId': '7dc39394-a63a-4b22-b799-7d36c954396d', 'address': {'slot': '0x02', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'qxl', 'type': 'video'}, {'device': 'spice', 'specParams': {'copyPasteEnable': 'true', 'displayNetwork': 'rhevm', 'keyMap': 'en-us', 'displayIp': '10.66.100.109', 'spiceSecureChannels': 'smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard'}, 'type': 'graphics', 'deviceType': 'graphics', 'tlsPort': '5900'}, {'nicModel': 'pv', 'macAddr': '00:1a:4a:ad:8d:00', 'linkActive': True, 'network': 'rhevm', 'alias': 'net0', 'bootOrder': '1', 'filter': 'vdsm-no-mac-spoofing', 'specParams': {'inbound': {}, 'outbound': {}}, 'deviceType': 'interface', 'deviceId': 'd438dab9-542d-4077-8515-f30b69256612', 'address': {'slot': '0x07', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface', 'name': 'vnet0'}, {'index': '2', 'iface': 'ide', 'name': 'hdc', 'alias': 'ide0-1-0', 'specParams': {'path': ''}, 'readonly': 'True', 'deviceType': 'disk', 'deviceId': 'f42525d0-86e2-43e6-a68e-0565b55dd504', 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'cdrom', 'shared': 'false', 'path': '', 'type': 'disk'}, {'address': {'slot': '0x05', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'volumeInfo': {'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610'}, 'index': 0, 'iface': 'virtio', 'apparentsize': '10737418240', 'alias': 'virtio-disk0', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610', 'readonly': 'False', 'deviceType': 'disk', 'shared': 'false', 'truesize': '4116598784', 'type': 'disk', 'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'reqsize': '0', 'format': 'raw', 'deviceId': 'a5611808-d282-45f9-9771-c3797b521610', 'poolID': '00000002-0002-0002-0002-0000000003e1', 'device': 'disk', 'path': u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'propagateErrors': 'off', 'optional': 'false', 'name': 'vda', 'bootOrder': '2', 'volumeID': '486f9fc4-3108-49db-ae60-4dc85544e8eb', 'specParams': {}, 'volumeChain': [{'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610'}]}], 'status': 'Migration Destination', 'timeOffset': '0', 'maxVCpus': '16', 'guestIPs': '', 'display': 'qxl'}} >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:59,141::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:59,142::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1047::DEBUG::2014-10-15 07:10:59,143::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:10:59,145::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:10:59,145::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1048::DEBUG::2014-10-15 07:10:59,146::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >libvirtEventLoop::DEBUG::2014-10-15 07:10:59,446::vm::5461::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::event Started detail 1 opaque None >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:02,161::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:02,163::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1049::DEBUG::2014-10-15 07:11:02,165::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:02,168::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:02,169::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1050::DEBUG::2014-10-15 07:11:02,171::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:05,184::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:05,185::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1051::DEBUG::2014-10-15 07:11:05,186::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:05,187::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:05,188::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1052::DEBUG::2014-10-15 07:11:05,188::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:11:07,762::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:07,763::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:07,763::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:07,763::__init__::232::IOProcess::(_processLogs) (902) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:11:07,764::__init__::232::IOProcess::(_processLogs) (902) Queuing response >Thread-25::DEBUG::2014-10-15 07:11:07,764::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:11:07,772::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000540184 s, 850 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:11:07,773::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:07,773::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:07,773::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:07,773::__init__::232::IOProcess::(_processLogs) (903) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:11:07,773::__init__::232::IOProcess::(_processLogs) (903) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:07,774::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:07,774::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:07,774::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:07,774::__init__::232::IOProcess::(_processLogs) (904) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:07,774::__init__::232::IOProcess::(_processLogs) (904) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:07,775::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:07,775::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:07,775::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:07,775::__init__::232::IOProcess::(_processLogs) (905) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:07,775::__init__::232::IOProcess::(_processLogs) (905) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:07,775::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:07,775::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:07,776::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:07,776::__init__::232::IOProcess::(_processLogs) (906) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:07,776::__init__::232::IOProcess::(_processLogs) (906) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:08,207::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:08,207::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1053::DEBUG::2014-10-15 07:11:08,209::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:08,210::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:08,211::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1054::DEBUG::2014-10-15 07:11:08,212::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:11,223::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:11,224::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1055::DEBUG::2014-10-15 07:11:11,225::task::595::Storage.TaskManager.Task::(_updateState) Task=`6361d759-9ef9-4081-8471-dda2a4d06aef`::moving from state init -> state preparing >Thread-1055::INFO::2014-10-15 07:11:11,225::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1055::INFO::2014-10-15 07:11:11,226::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000540184', 'lastCheck': '3.4', 'valid': True}} >Thread-1055::DEBUG::2014-10-15 07:11:11,226::task::1191::Storage.TaskManager.Task::(prepare) Task=`6361d759-9ef9-4081-8471-dda2a4d06aef`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000540184', 'lastCheck': '3.4', 'valid': True}} >Thread-1055::DEBUG::2014-10-15 07:11:11,226::task::595::Storage.TaskManager.Task::(_updateState) Task=`6361d759-9ef9-4081-8471-dda2a4d06aef`::moving from state preparing -> state finished >Thread-1055::DEBUG::2014-10-15 07:11:11,226::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1055::DEBUG::2014-10-15 07:11:11,226::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1055::DEBUG::2014-10-15 07:11:11,226::task::993::Storage.TaskManager.Task::(_decref) Task=`6361d759-9ef9-4081-8471-dda2a4d06aef`::ref 0 aborting False >Thread-1055::ERROR::2014-10-15 07:11:11,227::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1055::DEBUG::2014-10-15 07:11:11,228::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:11,239::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:11,239::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1056::DEBUG::2014-10-15 07:11:11,240::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:14,300::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:14,301::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1057::DEBUG::2014-10-15 07:11:14,302::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:14,303::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:14,304::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1058::DEBUG::2014-10-15 07:11:14,304::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:17,313::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:17,314::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1059::DEBUG::2014-10-15 07:11:17,315::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:17,316::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:17,316::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1060::DEBUG::2014-10-15 07:11:17,317::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:11:17,777::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:17,777::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:17,777::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:17,777::__init__::232::IOProcess::(_processLogs) (907) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:11:17,778::__init__::232::IOProcess::(_processLogs) (907) Queuing response >Thread-25::DEBUG::2014-10-15 07:11:17,778::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:11:17,784::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000571705 s, 803 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:11:17,785::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:17,785::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:17,785::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:17,786::__init__::232::IOProcess::(_processLogs) (908) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:11:17,786::__init__::232::IOProcess::(_processLogs) (908) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:17,786::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:17,786::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:17,787::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:17,787::__init__::232::IOProcess::(_processLogs) (909) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:17,787::__init__::232::IOProcess::(_processLogs) (909) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:17,787::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:17,787::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:17,787::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:17,788::__init__::232::IOProcess::(_processLogs) (910) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:17,788::__init__::232::IOProcess::(_processLogs) (910) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:17,788::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:17,788::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:17,788::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:17,788::__init__::232::IOProcess::(_processLogs) (911) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:17,788::__init__::232::IOProcess::(_processLogs) (911) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:20,328::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:20,329::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1061::DEBUG::2014-10-15 07:11:20,330::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:20,331::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:20,331::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1062::DEBUG::2014-10-15 07:11:20,332::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:23,344::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:23,344::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1063::DEBUG::2014-10-15 07:11:23,345::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:23,347::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:23,347::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1064::DEBUG::2014-10-15 07:11:23,348::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >libvirtEventLoop::DEBUG::2014-10-15 07:11:26,048::vm::5461::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::event Resumed detail 0 opaque None >libvirtEventLoop::DEBUG::2014-10-15 07:11:26,049::vm::5461::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::event Resumed detail 1 opaque None >Thread-1045::DEBUG::2014-10-15 07:11:26,081::vm::5301::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Looking for drive with attributes {'name': u'hdc', 'bootOrder': '', 'boot': [], 'readonly': True, 'address': {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, 'path': '', 'type': u'cdrom'} >Thread-1045::DEBUG::2014-10-15 07:11:26,081::vm::5322::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'hdc', u'hdc'), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, True), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}), 'path': ('', ''), 'type': (u'cdrom', u'cdrom')} >Thread-1045::DEBUG::2014-10-15 07:11:26,081::vm::5341::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'hdc', None), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, None), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, None), 'path': ('', None), 'type': (u'cdrom', None)} >Thread-1045::DEBUG::2014-10-15 07:11:26,082::vm::5301::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Looking for drive with attributes {'name': u'vda', 'bootOrder': u'2', 'boot': [<DOM Element: boot at 0x7f09340c0170>], 'readonly': False, 'address': {u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, 'path': u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'type': u'disk'} >Thread-1045::DEBUG::2014-10-15 07:11:26,082::vm::5322::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'vda', u'vda'), 'bootOrder': (u'2', u'2'), 'boot': ([<DOM Element: boot at 0x7f09340c0170>], None), 'readonly': (False, False), 'address': ({u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, {u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}), 'path': (u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb'), 'type': (u'disk', u'disk')} >Thread-1045::DEBUG::2014-10-15 07:11:26,082::vm::5341::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'vda', None), 'bootOrder': (u'2', None), 'boot': ([<DOM Element: boot at 0x7f09340c0170>], None), 'readonly': (False, None), 'address': ({u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, None), 'path': (u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', None), 'type': (u'disk', None)} >Thread-1045::DEBUG::2014-10-15 07:11:26,116::sampling::405::vm.Vm::(start) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Start statistics collection >Thread-1065::DEBUG::2014-10-15 07:11:26,116::sampling::434::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stats thread started >Thread-1065::DEBUG::2014-10-15 07:11:26,118::task::595::Storage.TaskManager.Task::(_updateState) Task=`c0b7890f-00c2-4859-a307-4d91c864a380`::moving from state init -> state preparing >Thread-1065::INFO::2014-10-15 07:11:26,118::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', volUUID='486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:11:26,119::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-1045::DEBUG::2014-10-15 07:11:26,119::vmchannels::199::vds::(register) Add fileno 30 to listener's channels. >Thread-16::DEBUG::2014-10-15 07:11:26,119::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:26,119::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:26,120::__init__::232::IOProcess::(_processLogs) (912) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:11:26,120::__init__::232::IOProcess::(_processLogs) (912) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:26,120::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:26,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:26,120::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:26,120::__init__::232::IOProcess::(_processLogs) (913) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:11:26,121::__init__::232::IOProcess::(_processLogs) (913) Queuing response >Thread-1065::INFO::2014-10-15 07:11:26,121::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:11:26,121::task::1191::Storage.TaskManager.Task::(prepare) Task=`c0b7890f-00c2-4859-a307-4d91c864a380`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:11:26,121::task::595::Storage.TaskManager.Task::(_updateState) Task=`c0b7890f-00c2-4859-a307-4d91c864a380`::moving from state preparing -> state finished >Thread-1065::DEBUG::2014-10-15 07:11:26,121::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1065::DEBUG::2014-10-15 07:11:26,121::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1065::DEBUG::2014-10-15 07:11:26,121::task::993::Storage.TaskManager.Task::(_decref) Task=`c0b7890f-00c2-4859-a307-4d91c864a380`::ref 0 aborting False >Thread-1045::DEBUG::2014-10-15 07:11:26,134::vm::4017::vm.Vm::(_completeIncomingMigration) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::End of migration >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:26,364::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:26,364::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1066::DEBUG::2014-10-15 07:11:26,365::task::595::Storage.TaskManager.Task::(_updateState) Task=`85eff55c-214b-4f98-9b7b-2bd5f0283ec4`::moving from state init -> state preparing >Thread-1066::INFO::2014-10-15 07:11:26,365::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1066::INFO::2014-10-15 07:11:26,366::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000571705', 'lastCheck': '8.6', 'valid': True}} >Thread-1066::DEBUG::2014-10-15 07:11:26,366::task::1191::Storage.TaskManager.Task::(prepare) Task=`85eff55c-214b-4f98-9b7b-2bd5f0283ec4`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000571705', 'lastCheck': '8.6', 'valid': True}} >Thread-1066::DEBUG::2014-10-15 07:11:26,366::task::595::Storage.TaskManager.Task::(_updateState) Task=`85eff55c-214b-4f98-9b7b-2bd5f0283ec4`::moving from state preparing -> state finished >Thread-1066::DEBUG::2014-10-15 07:11:26,366::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1066::DEBUG::2014-10-15 07:11:26,366::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1066::DEBUG::2014-10-15 07:11:26,366::task::993::Storage.TaskManager.Task::(_decref) Task=`85eff55c-214b-4f98-9b7b-2bd5f0283ec4`::ref 0 aborting False >Thread-1066::ERROR::2014-10-15 07:11:26,367::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1066::DEBUG::2014-10-15 07:11:26,368::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:26,379::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:26,379::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1067::DEBUG::2014-10-15 07:11:26,380::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:26,396::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-1068::DEBUG::2014-10-15 07:11:26,396::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getVMFullList' in bridge with {u'vmList': [u'c255610f-42c2-426d-9270-e4497f4a0f37']} >JsonRpcServer::DEBUG::2014-10-15 07:11:26,396::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1068::WARNING::2014-10-15 07:11:26,397::Bridge::212::root::(_typeFixup) Fixing up type VmDevice >Thread-1068::WARNING::2014-10-15 07:11:26,397::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-1068::WARNING::2014-10-15 07:11:26,398::Bridge::212::root::(_typeFixup) Fixing up type VmDevice >Thread-1068::WARNING::2014-10-15 07:11:26,398::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-1068::WARNING::2014-10-15 07:11:26,398::Bridge::212::root::(_typeFixup) Fixing up type VmDevice >Thread-1068::WARNING::2014-10-15 07:11:26,398::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-1068::WARNING::2014-10-15 07:11:26,398::Bridge::212::root::(_typeFixup) Fixing up type VmDevice >Thread-1068::WARNING::2014-10-15 07:11:26,398::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-1068::WARNING::2014-10-15 07:11:26,398::Bridge::212::root::(_typeFixup) Fixing up type VmDevice >Thread-1068::WARNING::2014-10-15 07:11:26,398::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-1068::WARNING::2014-10-15 07:11:26,399::Bridge::212::root::(_typeFixup) Fixing up type VmDevice >Thread-1068::WARNING::2014-10-15 07:11:26,399::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-1068::WARNING::2014-10-15 07:11:26,399::Bridge::212::root::(_typeFixup) Fixing up type VmDevice >Thread-1068::WARNING::2014-10-15 07:11:26,399::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-1068::WARNING::2014-10-15 07:11:26,399::Bridge::212::root::(_typeFixup) Fixing up type VmDevice >Thread-1068::WARNING::2014-10-15 07:11:26,399::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-1068::WARNING::2014-10-15 07:11:26,399::Bridge::212::root::(_typeFixup) Fixing up type VmDevice >Thread-1068::WARNING::2014-10-15 07:11:26,399::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >VM Channels Listener::DEBUG::2014-10-15 07:11:26,400::vmchannels::107::vds::(_do_add_channels) fileno 30 was added to unconnected channels. >VM Channels Listener::DEBUG::2014-10-15 07:11:26,400::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 30. >Thread-1068::WARNING::2014-10-15 07:11:26,400::Bridge::212::root::(_typeFixup) Fixing up type VmDevice >VM Channels Listener::DEBUG::2014-10-15 07:11:26,401::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >Thread-1068::WARNING::2014-10-15 07:11:26,401::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-1068::WARNING::2014-10-15 07:11:26,401::Bridge::212::root::(_typeFixup) Fixing up type VmDevice >VM Channels Listener::DEBUG::2014-10-15 07:11:26,402::guestagent::203::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Connected to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >Thread-1068::WARNING::2014-10-15 07:11:26,402::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >VM Channels Listener::DEBUG::2014-10-15 07:11:26,402::vmchannels::149::vds::(_handle_unconnected) Connecting to fileno 30 succeeded. >Thread-1068::WARNING::2014-10-15 07:11:26,402::Bridge::212::root::(_typeFixup) Fixing up type VmDevice >Thread-1068::WARNING::2014-10-15 07:11:26,403::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-1068::WARNING::2014-10-15 07:11:26,403::Bridge::212::root::(_typeFixup) Fixing up type VmDevice >Thread-1068::WARNING::2014-10-15 07:11:26,403::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-1068::DEBUG::2014-10-15 07:11:26,403::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getVMFullList' in bridge with [{'acpiEnable': 'true', 'emulatedMachine': 'rhel6.5.0', 'afterMigrationStatus': '', 'vmId': 'c255610f-42c2-426d-9270-e4497f4a0f37', 'memGuaranteedSize': 1024, 'transparentHugePages': 'true', 'displaySecurePort': u'5900', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'SandyBridge', 'smp': '1', 'custom': {'device_f47764b9-9bdd-497e-aa23-ac4b42744926device_8637879d-080d-4b8e-b157-43a915564601device_72fc718b-c615-400d-981a-ad80b846f515device_d45c1a74-983c-4785-b321-73e4f4191870': 'VmDevice {vmId=c255610f-42c2-426d-9270-e4497f4a0f37, deviceId=d45c1a74-983c-4785-b321-73e4f4191870, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=3}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}', 'device_f47764b9-9bdd-497e-aa23-ac4b42744926device_8637879d-080d-4b8e-b157-43a915564601': 'VmDevice {vmId=c255610f-42c2-426d-9270-e4497f4a0f37, deviceId=8637879d-080d-4b8e-b157-43a915564601, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=1}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}', 'device_f47764b9-9bdd-497e-aa23-ac4b42744926device_8637879d-080d-4b8e-b157-43a915564601device_72fc718b-c615-400d-981a-ad80b846f515': 'VmDevice {vmId=c255610f-42c2-426d-9270-e4497f4a0f37, deviceId=72fc718b-c615-400d-981a-ad80b846f515, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=2}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}', 'device_f47764b9-9bdd-497e-aa23-ac4b42744926': 'VmDevice {vmId=c255610f-42c2-426d-9270-e4497f4a0f37, deviceId=f47764b9-9bdd-497e-aa23-ac4b42744926, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}'}, 'vmType': 'kvm', 'memSize': 1024, 'smpCoresPerSocket': '1', 'vmName': 'rhevh-guest-rhel7.0', 'nice': '0', 'status': 'Up', 'bootMenuEnable': 'false', 'pid': '17954', 'copyPasteEnable': 'true', 'displayIp': '10.66.100.109', 'displayPort': '-1', 'smartcardEnable': 'false', 'spiceSecureChannels': 'smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard', 'fileTransferEnable': 'true', 'nicModel': 'rtl8139,pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'displayNetwork': 'rhevm', 'devices': [{'target': 1048576, 'alias': 'balloon0', 'specParams': {'model': 'virtio'}, 'deviceType': 'balloon', 'deviceId': '85359974-17aa-43a3-a43a-b66ead768ca8', 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'memballoon', 'type': 'balloon'}, {'device': 'unix', 'alias': 'channel0', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '1'}, 'deviceType': 'channel', 'type': 'channel'}, {'device': 'unix', 'alias': 'channel1', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '2'}, 'deviceType': 'channel', 'type': 'channel'}, {'device': 'spicevmc', 'alias': 'channel2', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '3'}, 'deviceType': 'channel', 'type': 'channel'}, {'index': '0', 'alias': u'scsi0', 'specParams': {}, 'deviceType': 'controller', 'deviceId': 'f6a83fc4-d8ba-4522-851c-112c295ac642', 'address': {u'slot': u'0x03', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, 'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}, {'alias': u'virtio-serial0', 'specParams': {}, 'deviceType': 'controller', 'deviceId': '5430e069-f469-4ff7-a9c4-e234e970951d', 'address': {u'slot': u'0x04', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, 'device': 'virtio-serial', 'type': 'controller'}, {'device': 'usb', 'alias': u'usb0', 'address': {u'slot': u'0x01', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x2'}, 'deviceType': 'controller', 'type': 'controller'}, {'device': 'ide', 'alias': u'ide0', 'address': {u'slot': u'0x01', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x1'}, 'deviceType': 'controller', 'type': 'controller'}, {'alias': 'video0', 'specParams': {'vram': '32768', 'heads': '1'}, 'deviceType': 'video', 'deviceId': '7dc39394-a63a-4b22-b799-7d36c954396d', 'address': {'slot': '0x02', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'qxl', 'type': 'video'}, {'device': 'spice', 'specParams': {'copyPasteEnable': 'true', 'displayNetwork': 'rhevm', 'keyMap': 'en-us', 'displayIp': '10.66.100.109', 'spiceSecureChannels': 'smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard'}, 'type': 'graphics', 'deviceType': 'graphics', 'tlsPort': u'5900'}, {'nicModel': 'pv', 'macAddr': '00:1a:4a:ad:8d:00', 'linkActive': True, 'network': 'rhevm', 'alias': u'net0', 'bootOrder': '1', 'filter': 'vdsm-no-mac-spoofing', 'specParams': {'inbound': {}, 'outbound': {}}, 'deviceType': 'interface', 'deviceId': 'd438dab9-542d-4077-8515-f30b69256612', 'address': {u'slot': u'0x07', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, 'device': 'bridge', 'type': 'interface', 'name': u'vnet0'}, {'index': '2', 'iface': 'ide', 'name': u'hdc', 'alias': u'ide0-1-0', 'specParams': {'path': ''}, 'readonly': 'True', 'deviceType': 'disk', 'deviceId': 'f42525d0-86e2-43e6-a68e-0565b55dd504', 'address': {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, 'device': 'cdrom', 'shared': 'false', 'path': '', 'type': 'disk'}, {'address': {u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, 'volumeInfo': {'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610'}, 'index': 0, 'iface': 'virtio', 'apparentsize': '10737418240', 'alias': u'virtio-disk0', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610', 'readonly': 'False', 'deviceType': 'disk', 'shared': 'false', 'truesize': '4116598784', 'type': 'disk', 'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'reqsize': '0', 'format': 'raw', 'deviceId': 'a5611808-d282-45f9-9771-c3797b521610', 'poolID': '00000002-0002-0002-0002-0000000003e1', 'device': 'disk', 'path': u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'propagateErrors': 'off', 'optional': 'false', 'name': u'vda', 'bootOrder': u'2', 'volumeID': '486f9fc4-3108-49db-ae60-4dc85544e8eb', 'specParams': {}, 'volumeChain': [{'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610'}]}], 'timeOffset': '0', 'maxVCpus': '16', 'clientIp': '', 'display': 'qxl'}] >Thread-1068::DEBUG::2014-10-15 07:11:26,404::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:11:27,789::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:27,789::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:27,790::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:27,790::__init__::232::IOProcess::(_processLogs) (914) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:11:27,790::__init__::232::IOProcess::(_processLogs) (914) Queuing response >Thread-25::DEBUG::2014-10-15 07:11:27,790::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:11:27,797::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000360883 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:11:27,797::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:27,797::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:27,798::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:27,798::__init__::232::IOProcess::(_processLogs) (915) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:11:27,798::__init__::232::IOProcess::(_processLogs) (915) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:27,798::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:27,798::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:27,798::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:27,799::__init__::232::IOProcess::(_processLogs) (916) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:27,799::__init__::232::IOProcess::(_processLogs) (916) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:27,799::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:27,799::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:27,799::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:27,799::__init__::232::IOProcess::(_processLogs) (917) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:27,799::__init__::232::IOProcess::(_processLogs) (917) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:27,800::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:27,800::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:27,800::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:27,800::__init__::232::IOProcess::(_processLogs) (918) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:27,800::__init__::232::IOProcess::(_processLogs) (918) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:29,597::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:29,598::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1069::DEBUG::2014-10-15 07:11:29,599::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:29,600::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:29,601::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1070::DEBUG::2014-10-15 07:11:29,602::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:32,631::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:32,632::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1071::DEBUG::2014-10-15 07:11:32,633::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:32,634::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:32,634::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1072::DEBUG::2014-10-15 07:11:32,636::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:35,653::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:35,654::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1073::DEBUG::2014-10-15 07:11:35,655::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:35,656::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:35,657::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1074::DEBUG::2014-10-15 07:11:35,658::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:11:37,801::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:37,801::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:37,801::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:37,802::__init__::232::IOProcess::(_processLogs) (919) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:11:37,802::__init__::232::IOProcess::(_processLogs) (919) Queuing response >Thread-25::DEBUG::2014-10-15 07:11:37,802::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:11:37,809::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000738883 s, 621 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:11:37,809::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:37,810::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:37,810::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:37,810::__init__::232::IOProcess::(_processLogs) (920) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:11:37,810::__init__::232::IOProcess::(_processLogs) (920) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:37,811::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:37,811::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:37,811::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:37,811::__init__::232::IOProcess::(_processLogs) (921) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:37,811::__init__::232::IOProcess::(_processLogs) (921) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:37,811::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:37,811::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:37,812::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:37,812::__init__::232::IOProcess::(_processLogs) (922) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:37,812::__init__::232::IOProcess::(_processLogs) (922) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:37,812::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:37,812::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:37,812::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:37,812::__init__::232::IOProcess::(_processLogs) (923) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:37,812::__init__::232::IOProcess::(_processLogs) (923) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:38,678::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:38,679::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1075::DEBUG::2014-10-15 07:11:38,680::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:38,681::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:38,682::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1076::DEBUG::2014-10-15 07:11:38,683::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:41,711::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:41,711::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1077::DEBUG::2014-10-15 07:11:41,713::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac3ac9fe-a1de-48a1-af0b-b3019a33d661`::moving from state init -> state preparing >Thread-1077::INFO::2014-10-15 07:11:41,713::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1077::INFO::2014-10-15 07:11:41,713::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000738883', 'lastCheck': '3.9', 'valid': True}} >Thread-1077::DEBUG::2014-10-15 07:11:41,713::task::1191::Storage.TaskManager.Task::(prepare) Task=`ac3ac9fe-a1de-48a1-af0b-b3019a33d661`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000738883', 'lastCheck': '3.9', 'valid': True}} >Thread-1077::DEBUG::2014-10-15 07:11:41,713::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac3ac9fe-a1de-48a1-af0b-b3019a33d661`::moving from state preparing -> state finished >Thread-1077::DEBUG::2014-10-15 07:11:41,713::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1077::DEBUG::2014-10-15 07:11:41,714::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1077::DEBUG::2014-10-15 07:11:41,714::task::993::Storage.TaskManager.Task::(_decref) Task=`ac3ac9fe-a1de-48a1-af0b-b3019a33d661`::ref 0 aborting False >Thread-1077::ERROR::2014-10-15 07:11:41,715::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1077::DEBUG::2014-10-15 07:11:41,715::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:41,728::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:41,729::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1078::DEBUG::2014-10-15 07:11:41,730::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1078::DEBUG::2014-10-15 07:11:41,730::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:44,849::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:44,850::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1079::DEBUG::2014-10-15 07:11:44,851::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:44,852::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:44,852::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1080::DEBUG::2014-10-15 07:11:44,853::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1080::DEBUG::2014-10-15 07:11:44,854::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:11:47,813::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:47,814::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:47,814::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:47,814::__init__::232::IOProcess::(_processLogs) (924) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:11:47,815::__init__::232::IOProcess::(_processLogs) (924) Queuing response >Thread-25::DEBUG::2014-10-15 07:11:47,815::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:11:47,822::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000595975 s, 770 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:11:47,822::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:47,822::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:47,822::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:47,823::__init__::232::IOProcess::(_processLogs) (925) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:11:47,823::__init__::232::IOProcess::(_processLogs) (925) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:47,823::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:47,823::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:47,824::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:47,824::__init__::232::IOProcess::(_processLogs) (926) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:47,824::__init__::232::IOProcess::(_processLogs) (926) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:47,824::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:47,824::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:47,824::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:47,825::__init__::232::IOProcess::(_processLogs) (927) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:47,825::__init__::232::IOProcess::(_processLogs) (927) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:47,825::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:47,825::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:47,825::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:47,825::__init__::232::IOProcess::(_processLogs) (928) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:47,825::__init__::232::IOProcess::(_processLogs) (928) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:47,914::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:47,915::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1081::DEBUG::2014-10-15 07:11:47,916::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:47,917::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:47,918::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1082::DEBUG::2014-10-15 07:11:47,919::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1082::DEBUG::2014-10-15 07:11:47,919::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:11:49,029::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:11:49,030::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:11:49,030::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:50,939::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:50,940::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1083::DEBUG::2014-10-15 07:11:50,941::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:50,942::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:50,943::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1084::DEBUG::2014-10-15 07:11:50,944::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1084::DEBUG::2014-10-15 07:11:50,944::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:53,958::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:53,958::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1085::DEBUG::2014-10-15 07:11:53,959::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:53,961::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:53,961::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1086::DEBUG::2014-10-15 07:11:53,962::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1086::DEBUG::2014-10-15 07:11:53,962::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:11:54,033::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:11:54,033::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:11:54,033::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:11:56,435::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:56,980::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:56,981::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1087::DEBUG::2014-10-15 07:11:56,982::task::595::Storage.TaskManager.Task::(_updateState) Task=`51a41127-6914-4299-92e8-0795a73d3e9c`::moving from state init -> state preparing >Thread-1087::INFO::2014-10-15 07:11:56,983::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1087::INFO::2014-10-15 07:11:56,983::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000595975', 'lastCheck': '9.2', 'valid': True}} >Thread-1087::DEBUG::2014-10-15 07:11:56,983::task::1191::Storage.TaskManager.Task::(prepare) Task=`51a41127-6914-4299-92e8-0795a73d3e9c`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000595975', 'lastCheck': '9.2', 'valid': True}} >Thread-1087::DEBUG::2014-10-15 07:11:56,983::task::595::Storage.TaskManager.Task::(_updateState) Task=`51a41127-6914-4299-92e8-0795a73d3e9c`::moving from state preparing -> state finished >Thread-1087::DEBUG::2014-10-15 07:11:56,983::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1087::DEBUG::2014-10-15 07:11:56,983::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1087::DEBUG::2014-10-15 07:11:56,983::task::993::Storage.TaskManager.Task::(_decref) Task=`51a41127-6914-4299-92e8-0795a73d3e9c`::ref 0 aborting False >Thread-1087::ERROR::2014-10-15 07:11:56,984::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1087::DEBUG::2014-10-15 07:11:56,985::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:11:56,994::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:11:56,994::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1088::DEBUG::2014-10-15 07:11:56,995::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1088::DEBUG::2014-10-15 07:11:56,996::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:11:57,826::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:57,827::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:57,827::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:57,827::__init__::232::IOProcess::(_processLogs) (929) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:11:57,827::__init__::232::IOProcess::(_processLogs) (929) Queuing response >Thread-25::DEBUG::2014-10-15 07:11:57,827::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:11:57,833::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000456126 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:11:57,834::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:57,834::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:57,834::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:57,834::__init__::232::IOProcess::(_processLogs) (930) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:11:57,834::__init__::232::IOProcess::(_processLogs) (930) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:57,835::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:57,835::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:57,835::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:57,835::__init__::232::IOProcess::(_processLogs) (931) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:57,835::__init__::232::IOProcess::(_processLogs) (931) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:57,836::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:57,836::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:57,836::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:57,836::__init__::232::IOProcess::(_processLogs) (932) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:57,836::__init__::232::IOProcess::(_processLogs) (932) Queuing response >Thread-16::DEBUG::2014-10-15 07:11:57,836::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:11:57,837::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:11:57,837::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:11:57,837::__init__::232::IOProcess::(_processLogs) (933) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:11:57,837::__init__::232::IOProcess::(_processLogs) (933) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:11:59,039::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:11:59,040::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:11:59,040::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:00,131::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:00,132::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1089::DEBUG::2014-10-15 07:12:00,133::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:00,134::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:00,134::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1090::DEBUG::2014-10-15 07:12:00,135::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1090::DEBUG::2014-10-15 07:12:00,135::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:03,166::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:03,167::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1091::DEBUG::2014-10-15 07:12:03,168::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:03,169::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:03,170::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1092::DEBUG::2014-10-15 07:12:03,170::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1092::DEBUG::2014-10-15 07:12:03,171::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:04,046::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:04,046::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:04,047::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:06,191::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:06,192::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1093::DEBUG::2014-10-15 07:12:06,193::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:06,194::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:06,194::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1094::DEBUG::2014-10-15 07:12:06,195::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1094::DEBUG::2014-10-15 07:12:06,195::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:12:07,838::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:07,838::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:07,838::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:07,838::__init__::232::IOProcess::(_processLogs) (934) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:12:07,839::__init__::232::IOProcess::(_processLogs) (934) Queuing response >Thread-25::DEBUG::2014-10-15 07:12:07,839::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:12:07,846::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000445347 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:12:07,847::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:07,847::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:07,847::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:07,848::__init__::232::IOProcess::(_processLogs) (935) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:12:07,848::__init__::232::IOProcess::(_processLogs) (935) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:07,848::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:07,848::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:07,848::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:07,849::__init__::232::IOProcess::(_processLogs) (936) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:07,849::__init__::232::IOProcess::(_processLogs) (936) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:07,849::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:07,849::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:07,849::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:07,849::__init__::232::IOProcess::(_processLogs) (937) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:07,849::__init__::232::IOProcess::(_processLogs) (937) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:07,850::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:07,850::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:07,850::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:07,850::__init__::232::IOProcess::(_processLogs) (938) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:07,850::__init__::232::IOProcess::(_processLogs) (938) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:09,049::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:09,050::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:09,050::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:09,210::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:09,211::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1095::DEBUG::2014-10-15 07:12:09,212::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:09,213::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:09,214::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1096::DEBUG::2014-10-15 07:12:09,214::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1096::DEBUG::2014-10-15 07:12:09,215::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:12,226::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:12,227::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1097::DEBUG::2014-10-15 07:12:12,228::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc30ceee-a606-4c4f-89be-52c8d1f134a7`::moving from state init -> state preparing >Thread-1097::INFO::2014-10-15 07:12:12,228::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1097::INFO::2014-10-15 07:12:12,229::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000445347', 'lastCheck': '4.4', 'valid': True}} >Thread-1097::DEBUG::2014-10-15 07:12:12,229::task::1191::Storage.TaskManager.Task::(prepare) Task=`cc30ceee-a606-4c4f-89be-52c8d1f134a7`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000445347', 'lastCheck': '4.4', 'valid': True}} >Thread-1097::DEBUG::2014-10-15 07:12:12,229::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc30ceee-a606-4c4f-89be-52c8d1f134a7`::moving from state preparing -> state finished >Thread-1097::DEBUG::2014-10-15 07:12:12,229::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1097::DEBUG::2014-10-15 07:12:12,229::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1097::DEBUG::2014-10-15 07:12:12,229::task::993::Storage.TaskManager.Task::(_decref) Task=`cc30ceee-a606-4c4f-89be-52c8d1f134a7`::ref 0 aborting False >Thread-1097::ERROR::2014-10-15 07:12:12,230::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1097::DEBUG::2014-10-15 07:12:12,231::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:12,242::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:12,242::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1098::DEBUG::2014-10-15 07:12:12,243::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1098::DEBUG::2014-10-15 07:12:12,244::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:14,056::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:14,056::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:14,057::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:15,368::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:15,369::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1099::DEBUG::2014-10-15 07:12:15,370::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:15,371::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:15,371::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1100::DEBUG::2014-10-15 07:12:15,372::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1100::DEBUG::2014-10-15 07:12:15,373::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:12:17,851::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:17,852::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:17,852::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:17,852::__init__::232::IOProcess::(_processLogs) (939) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:12:17,852::__init__::232::IOProcess::(_processLogs) (939) Queuing response >Thread-25::DEBUG::2014-10-15 07:12:17,852::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:12:17,858::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000394991 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:12:17,859::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:17,859::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:17,859::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:17,859::__init__::232::IOProcess::(_processLogs) (940) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:12:17,859::__init__::232::IOProcess::(_processLogs) (940) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:17,860::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:17,860::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:17,860::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:17,860::__init__::232::IOProcess::(_processLogs) (941) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:17,860::__init__::232::IOProcess::(_processLogs) (941) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:17,861::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:17,861::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:17,861::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:17,861::__init__::232::IOProcess::(_processLogs) (942) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:17,861::__init__::232::IOProcess::(_processLogs) (942) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:17,861::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:17,861::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:17,862::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:17,862::__init__::232::IOProcess::(_processLogs) (943) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:17,862::__init__::232::IOProcess::(_processLogs) (943) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:18,416::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:18,417::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1101::DEBUG::2014-10-15 07:12:18,418::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:18,419::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:18,419::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1102::DEBUG::2014-10-15 07:12:18,420::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1102::DEBUG::2014-10-15 07:12:18,421::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:19,062::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:19,063::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:19,063::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:21,444::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:21,445::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1103::DEBUG::2014-10-15 07:12:21,446::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:21,448::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:21,448::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1104::DEBUG::2014-10-15 07:12:21,449::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1104::DEBUG::2014-10-15 07:12:21,449::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:24,069::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:24,069::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:24,070::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:24,471::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:24,472::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1105::DEBUG::2014-10-15 07:12:24,473::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:24,474::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:24,475::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1106::DEBUG::2014-10-15 07:12:24,475::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1106::DEBUG::2014-10-15 07:12:24,476::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-1065::DEBUG::2014-10-15 07:12:26,165::task::595::Storage.TaskManager.Task::(_updateState) Task=`b82268b1-b920-4b67-9afb-7be6c95ae04f`::moving from state init -> state preparing >Thread-1065::INFO::2014-10-15 07:12:26,165::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', volUUID='486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:12:26,166::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:26,166::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:26,166::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:26,166::__init__::232::IOProcess::(_processLogs) (944) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:12:26,167::__init__::232::IOProcess::(_processLogs) (944) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:26,167::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:26,167::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:26,168::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:26,168::__init__::232::IOProcess::(_processLogs) (945) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:12:26,168::__init__::232::IOProcess::(_processLogs) (945) Queuing response >Thread-1065::INFO::2014-10-15 07:12:26,168::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:12:26,168::task::1191::Storage.TaskManager.Task::(prepare) Task=`b82268b1-b920-4b67-9afb-7be6c95ae04f`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:12:26,168::task::595::Storage.TaskManager.Task::(_updateState) Task=`b82268b1-b920-4b67-9afb-7be6c95ae04f`::moving from state preparing -> state finished >Thread-1065::DEBUG::2014-10-15 07:12:26,168::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1065::DEBUG::2014-10-15 07:12:26,168::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1065::DEBUG::2014-10-15 07:12:26,169::task::993::Storage.TaskManager.Task::(_decref) Task=`b82268b1-b920-4b67-9afb-7be6c95ae04f`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 07:12:26,468::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:27,503::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:27,504::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1107::DEBUG::2014-10-15 07:12:27,505::task::595::Storage.TaskManager.Task::(_updateState) Task=`bcd78844-a93f-4a9d-a6f1-2bac31cb0890`::moving from state init -> state preparing >Thread-1107::INFO::2014-10-15 07:12:27,505::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1107::INFO::2014-10-15 07:12:27,505::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000394991', 'lastCheck': '9.6', 'valid': True}} >Thread-1107::DEBUG::2014-10-15 07:12:27,505::task::1191::Storage.TaskManager.Task::(prepare) Task=`bcd78844-a93f-4a9d-a6f1-2bac31cb0890`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000394991', 'lastCheck': '9.6', 'valid': True}} >Thread-1107::DEBUG::2014-10-15 07:12:27,505::task::595::Storage.TaskManager.Task::(_updateState) Task=`bcd78844-a93f-4a9d-a6f1-2bac31cb0890`::moving from state preparing -> state finished >Thread-1107::DEBUG::2014-10-15 07:12:27,506::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1107::DEBUG::2014-10-15 07:12:27,506::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1107::DEBUG::2014-10-15 07:12:27,506::task::993::Storage.TaskManager.Task::(_decref) Task=`bcd78844-a93f-4a9d-a6f1-2bac31cb0890`::ref 0 aborting False >Thread-1107::ERROR::2014-10-15 07:12:27,507::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1107::DEBUG::2014-10-15 07:12:27,507::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:27,516::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:27,517::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1108::DEBUG::2014-10-15 07:12:27,518::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1108::DEBUG::2014-10-15 07:12:27,518::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:12:27,863::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:27,863::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:27,863::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:27,863::__init__::232::IOProcess::(_processLogs) (946) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:12:27,863::__init__::232::IOProcess::(_processLogs) (946) Queuing response >Thread-25::DEBUG::2014-10-15 07:12:27,864::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:12:27,869::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000356624 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:12:27,870::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:27,870::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:27,870::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:27,871::__init__::232::IOProcess::(_processLogs) (947) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:12:27,871::__init__::232::IOProcess::(_processLogs) (947) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:27,871::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:27,871::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:27,871::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:27,871::__init__::232::IOProcess::(_processLogs) (948) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:27,872::__init__::232::IOProcess::(_processLogs) (948) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:27,872::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:27,872::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:27,872::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:27,872::__init__::232::IOProcess::(_processLogs) (949) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:27,872::__init__::232::IOProcess::(_processLogs) (949) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:27,873::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:27,873::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:27,873::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:27,873::__init__::232::IOProcess::(_processLogs) (950) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:27,873::__init__::232::IOProcess::(_processLogs) (950) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:29,076::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:29,076::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:29,077::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:30,687::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:30,688::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1109::DEBUG::2014-10-15 07:12:30,690::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:30,692::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:30,693::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1110::DEBUG::2014-10-15 07:12:30,695::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1110::DEBUG::2014-10-15 07:12:30,696::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:33,738::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:33,738::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1111::DEBUG::2014-10-15 07:12:33,740::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:33,740::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:33,741::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1112::DEBUG::2014-10-15 07:12:33,742::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1112::DEBUG::2014-10-15 07:12:33,742::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:34,082::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:34,083::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:34,083::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:36,754::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:36,755::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1113::DEBUG::2014-10-15 07:12:36,756::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:36,757::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:36,758::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1114::DEBUG::2014-10-15 07:12:36,759::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1114::DEBUG::2014-10-15 07:12:36,759::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:12:37,874::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:37,874::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:37,874::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:37,875::__init__::232::IOProcess::(_processLogs) (951) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:12:37,875::__init__::232::IOProcess::(_processLogs) (951) Queuing response >Thread-25::DEBUG::2014-10-15 07:12:37,875::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:12:37,881::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000461664 s, 994 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:12:37,882::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:37,882::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:37,882::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:37,882::__init__::232::IOProcess::(_processLogs) (952) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:12:37,882::__init__::232::IOProcess::(_processLogs) (952) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:37,883::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:37,883::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:37,883::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:37,883::__init__::232::IOProcess::(_processLogs) (953) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:37,883::__init__::232::IOProcess::(_processLogs) (953) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:37,884::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:37,884::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:37,884::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:37,884::__init__::232::IOProcess::(_processLogs) (954) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:37,884::__init__::232::IOProcess::(_processLogs) (954) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:37,884::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:37,884::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:37,885::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:37,885::__init__::232::IOProcess::(_processLogs) (955) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:37,885::__init__::232::IOProcess::(_processLogs) (955) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:39,089::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:39,089::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:39,090::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:39,779::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:39,779::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1115::DEBUG::2014-10-15 07:12:39,780::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:39,782::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:39,782::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1116::DEBUG::2014-10-15 07:12:39,783::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1116::DEBUG::2014-10-15 07:12:39,783::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:42,806::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:42,807::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1117::DEBUG::2014-10-15 07:12:42,808::task::595::Storage.TaskManager.Task::(_updateState) Task=`cde351e0-56c7-45a0-9e6d-7129534d488f`::moving from state init -> state preparing >Thread-1117::INFO::2014-10-15 07:12:42,808::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1117::INFO::2014-10-15 07:12:42,809::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000461664', 'lastCheck': '4.9', 'valid': True}} >Thread-1117::DEBUG::2014-10-15 07:12:42,809::task::1191::Storage.TaskManager.Task::(prepare) Task=`cde351e0-56c7-45a0-9e6d-7129534d488f`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000461664', 'lastCheck': '4.9', 'valid': True}} >Thread-1117::DEBUG::2014-10-15 07:12:42,809::task::595::Storage.TaskManager.Task::(_updateState) Task=`cde351e0-56c7-45a0-9e6d-7129534d488f`::moving from state preparing -> state finished >Thread-1117::DEBUG::2014-10-15 07:12:42,809::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1117::DEBUG::2014-10-15 07:12:42,809::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1117::DEBUG::2014-10-15 07:12:42,809::task::993::Storage.TaskManager.Task::(_decref) Task=`cde351e0-56c7-45a0-9e6d-7129534d488f`::ref 0 aborting False >Thread-1117::ERROR::2014-10-15 07:12:42,810::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1117::DEBUG::2014-10-15 07:12:42,811::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:42,823::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:42,824::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1118::DEBUG::2014-10-15 07:12:42,825::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1118::DEBUG::2014-10-15 07:12:42,825::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:44,096::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:44,096::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:44,096::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:45,933::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:45,934::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1119::DEBUG::2014-10-15 07:12:45,936::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:45,937::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:45,937::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1120::DEBUG::2014-10-15 07:12:45,941::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1120::DEBUG::2014-10-15 07:12:45,942::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:12:47,886::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:47,886::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:47,886::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:47,886::__init__::232::IOProcess::(_processLogs) (956) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:12:47,886::__init__::232::IOProcess::(_processLogs) (956) Queuing response >Thread-25::DEBUG::2014-10-15 07:12:47,887::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:12:47,893::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000384121 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:12:47,894::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:47,894::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:47,894::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:47,894::__init__::232::IOProcess::(_processLogs) (957) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:12:47,894::__init__::232::IOProcess::(_processLogs) (957) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:47,895::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:47,895::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:47,895::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:47,895::__init__::232::IOProcess::(_processLogs) (958) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:47,895::__init__::232::IOProcess::(_processLogs) (958) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:47,895::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:47,895::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:47,896::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:47,896::__init__::232::IOProcess::(_processLogs) (959) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:47,896::__init__::232::IOProcess::(_processLogs) (959) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:47,896::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:47,896::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:47,896::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:47,896::__init__::232::IOProcess::(_processLogs) (960) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:47,897::__init__::232::IOProcess::(_processLogs) (960) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:49,076::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:49,077::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1121::DEBUG::2014-10-15 07:12:49,078::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:49,080::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:49,080::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1122::DEBUG::2014-10-15 07:12:49,081::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1122::DEBUG::2014-10-15 07:12:49,081::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:49,102::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:49,103::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:49,103::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:52,099::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:52,100::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1123::DEBUG::2014-10-15 07:12:52,101::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:52,102::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:52,103::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1124::DEBUG::2014-10-15 07:12:52,104::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1124::DEBUG::2014-10-15 07:12:52,104::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:54,107::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:54,108::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:54,108::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:55,117::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:55,117::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1125::DEBUG::2014-10-15 07:12:55,118::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:55,119::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:55,119::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1126::DEBUG::2014-10-15 07:12:55,120::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1126::DEBUG::2014-10-15 07:12:55,121::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:12:56,500::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >Thread-16::DEBUG::2014-10-15 07:12:57,898::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:57,898::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:57,898::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:57,898::__init__::232::IOProcess::(_processLogs) (961) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:12:57,898::__init__::232::IOProcess::(_processLogs) (961) Queuing response >Thread-25::DEBUG::2014-10-15 07:12:57,898::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:12:57,905::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000377754 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:12:57,905::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:57,905::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:57,906::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:57,906::__init__::232::IOProcess::(_processLogs) (962) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:12:57,906::__init__::232::IOProcess::(_processLogs) (962) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:57,906::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:57,907::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:57,907::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:57,907::__init__::232::IOProcess::(_processLogs) (963) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:57,907::__init__::232::IOProcess::(_processLogs) (963) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:57,907::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:57,907::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:57,907::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:57,907::__init__::232::IOProcess::(_processLogs) (964) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:57,908::__init__::232::IOProcess::(_processLogs) (964) Queuing response >Thread-16::DEBUG::2014-10-15 07:12:57,908::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:12:57,908::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:12:57,908::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:12:57,908::__init__::232::IOProcess::(_processLogs) (965) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:12:57,908::__init__::232::IOProcess::(_processLogs) (965) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:58,160::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:58,161::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1127::DEBUG::2014-10-15 07:12:58,162::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c869ad2-17ef-433d-b6ca-802b41c14712`::moving from state init -> state preparing >Thread-1127::INFO::2014-10-15 07:12:58,163::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1127::INFO::2014-10-15 07:12:58,163::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000377754', 'lastCheck': '0.3', 'valid': True}} >Thread-1127::DEBUG::2014-10-15 07:12:58,163::task::1191::Storage.TaskManager.Task::(prepare) Task=`6c869ad2-17ef-433d-b6ca-802b41c14712`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000377754', 'lastCheck': '0.3', 'valid': True}} >Thread-1127::DEBUG::2014-10-15 07:12:58,163::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c869ad2-17ef-433d-b6ca-802b41c14712`::moving from state preparing -> state finished >Thread-1127::DEBUG::2014-10-15 07:12:58,163::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1127::DEBUG::2014-10-15 07:12:58,163::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1127::DEBUG::2014-10-15 07:12:58,163::task::993::Storage.TaskManager.Task::(_decref) Task=`6c869ad2-17ef-433d-b6ca-802b41c14712`::ref 0 aborting False >Thread-1127::ERROR::2014-10-15 07:12:58,164::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1127::DEBUG::2014-10-15 07:12:58,165::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:12:58,176::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:12:58,177::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1128::DEBUG::2014-10-15 07:12:58,178::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1128::DEBUG::2014-10-15 07:12:58,178::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:59,113::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:59,114::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:12:59,114::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:01,307::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:01,308::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1129::DEBUG::2014-10-15 07:13:01,309::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:01,310::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:01,311::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1130::DEBUG::2014-10-15 07:13:01,311::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1130::DEBUG::2014-10-15 07:13:01,312::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:04,117::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:04,117::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:04,118::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:04,341::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:04,342::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1131::DEBUG::2014-10-15 07:13:04,342::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:04,343::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:04,344::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1132::DEBUG::2014-10-15 07:13:04,345::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1132::DEBUG::2014-10-15 07:13:04,345::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:07,370::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:07,370::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1133::DEBUG::2014-10-15 07:13:07,371::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:07,373::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:07,373::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1134::DEBUG::2014-10-15 07:13:07,374::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1134::DEBUG::2014-10-15 07:13:07,374::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:13:07,909::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:07,909::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:07,910::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:07,910::__init__::232::IOProcess::(_processLogs) (966) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:13:07,910::__init__::232::IOProcess::(_processLogs) (966) Queuing response >Thread-25::DEBUG::2014-10-15 07:13:07,911::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:13:07,918::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000428408 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:13:07,918::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:07,919::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:07,919::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:07,919::__init__::232::IOProcess::(_processLogs) (967) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:13:07,919::__init__::232::IOProcess::(_processLogs) (967) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:07,919::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:07,919::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:07,920::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:07,920::__init__::232::IOProcess::(_processLogs) (968) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:07,920::__init__::232::IOProcess::(_processLogs) (968) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:07,920::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:07,920::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:07,920::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:07,920::__init__::232::IOProcess::(_processLogs) (969) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:07,920::__init__::232::IOProcess::(_processLogs) (969) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:07,921::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:07,921::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:07,921::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:07,921::__init__::232::IOProcess::(_processLogs) (970) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:07,921::__init__::232::IOProcess::(_processLogs) (970) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:09,124::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:09,124::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:09,125::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:10,392::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:10,393::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1135::DEBUG::2014-10-15 07:13:10,394::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:10,395::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:10,396::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1136::DEBUG::2014-10-15 07:13:10,396::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1136::DEBUG::2014-10-15 07:13:10,397::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:13,418::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:13,419::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1137::DEBUG::2014-10-15 07:13:13,420::task::595::Storage.TaskManager.Task::(_updateState) Task=`e765264e-15a0-4474-8dab-fa82fdfd6a48`::moving from state init -> state preparing >Thread-1137::INFO::2014-10-15 07:13:13,420::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1137::INFO::2014-10-15 07:13:13,420::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428408', 'lastCheck': '5.5', 'valid': True}} >Thread-1137::DEBUG::2014-10-15 07:13:13,421::task::1191::Storage.TaskManager.Task::(prepare) Task=`e765264e-15a0-4474-8dab-fa82fdfd6a48`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428408', 'lastCheck': '5.5', 'valid': True}} >Thread-1137::DEBUG::2014-10-15 07:13:13,421::task::595::Storage.TaskManager.Task::(_updateState) Task=`e765264e-15a0-4474-8dab-fa82fdfd6a48`::moving from state preparing -> state finished >Thread-1137::DEBUG::2014-10-15 07:13:13,421::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1137::DEBUG::2014-10-15 07:13:13,421::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1137::DEBUG::2014-10-15 07:13:13,421::task::993::Storage.TaskManager.Task::(_decref) Task=`e765264e-15a0-4474-8dab-fa82fdfd6a48`::ref 0 aborting False >Thread-1137::ERROR::2014-10-15 07:13:13,422::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1137::DEBUG::2014-10-15 07:13:13,423::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:13,433::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:13,433::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1138::DEBUG::2014-10-15 07:13:13,434::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1138::DEBUG::2014-10-15 07:13:13,434::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:14,130::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:14,131::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:14,131::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:16,571::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:16,572::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1139::DEBUG::2014-10-15 07:13:16,573::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:16,574::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:16,574::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1140::DEBUG::2014-10-15 07:13:16,575::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1140::DEBUG::2014-10-15 07:13:16,576::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:13:17,922::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:17,922::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:17,922::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:17,923::__init__::232::IOProcess::(_processLogs) (971) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:13:17,923::__init__::232::IOProcess::(_processLogs) (971) Queuing response >Thread-25::DEBUG::2014-10-15 07:13:17,923::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:13:17,929::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000366315 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:13:17,930::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:17,930::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:17,930::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:17,930::__init__::232::IOProcess::(_processLogs) (972) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:13:17,930::__init__::232::IOProcess::(_processLogs) (972) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:17,931::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:17,931::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:17,931::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:17,931::__init__::232::IOProcess::(_processLogs) (973) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:17,931::__init__::232::IOProcess::(_processLogs) (973) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:17,932::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:17,932::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:17,932::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:17,932::__init__::232::IOProcess::(_processLogs) (974) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:17,932::__init__::232::IOProcess::(_processLogs) (974) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:17,932::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:17,932::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:17,932::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:17,933::__init__::232::IOProcess::(_processLogs) (975) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:17,933::__init__::232::IOProcess::(_processLogs) (975) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:19,137::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:19,138::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:19,138::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:19,606::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:19,607::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1141::DEBUG::2014-10-15 07:13:19,608::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:19,609::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:19,610::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1142::DEBUG::2014-10-15 07:13:19,610::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1142::DEBUG::2014-10-15 07:13:19,611::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:22,627::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:22,628::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1143::DEBUG::2014-10-15 07:13:22,629::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:22,630::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:22,631::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1144::DEBUG::2014-10-15 07:13:22,632::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1144::DEBUG::2014-10-15 07:13:22,632::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:24,141::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:24,141::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:24,142::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:25,656::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:25,657::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1145::DEBUG::2014-10-15 07:13:25,657::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:25,659::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:25,659::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1146::DEBUG::2014-10-15 07:13:25,660::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1146::DEBUG::2014-10-15 07:13:25,660::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-1065::DEBUG::2014-10-15 07:13:26,207::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb23e749-7fb3-426c-ae53-d1a5159c63a5`::moving from state init -> state preparing >Thread-1065::INFO::2014-10-15 07:13:26,208::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', volUUID='486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:13:26,208::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:26,208::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:26,208::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:26,209::__init__::232::IOProcess::(_processLogs) (976) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:13:26,209::__init__::232::IOProcess::(_processLogs) (976) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:26,210::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:26,210::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:26,210::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:26,210::__init__::232::IOProcess::(_processLogs) (977) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:13:26,210::__init__::232::IOProcess::(_processLogs) (977) Queuing response >Thread-1065::INFO::2014-10-15 07:13:26,210::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:13:26,211::task::1191::Storage.TaskManager.Task::(prepare) Task=`bb23e749-7fb3-426c-ae53-d1a5159c63a5`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:13:26,211::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb23e749-7fb3-426c-ae53-d1a5159c63a5`::moving from state preparing -> state finished >Thread-1065::DEBUG::2014-10-15 07:13:26,211::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1065::DEBUG::2014-10-15 07:13:26,211::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1065::DEBUG::2014-10-15 07:13:26,211::task::993::Storage.TaskManager.Task::(_decref) Task=`bb23e749-7fb3-426c-ae53-d1a5159c63a5`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 07:13:26,531::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >Thread-16::DEBUG::2014-10-15 07:13:27,933::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:27,934::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:27,934::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:27,934::__init__::232::IOProcess::(_processLogs) (978) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:13:27,934::__init__::232::IOProcess::(_processLogs) (978) Queuing response >Thread-25::DEBUG::2014-10-15 07:13:27,934::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:13:27,941::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000366136 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:13:27,941::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:27,941::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:27,941::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:27,942::__init__::232::IOProcess::(_processLogs) (979) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:13:27,942::__init__::232::IOProcess::(_processLogs) (979) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:27,942::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:27,942::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:27,943::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:27,943::__init__::232::IOProcess::(_processLogs) (980) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:27,943::__init__::232::IOProcess::(_processLogs) (980) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:27,943::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:27,943::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:27,943::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:27,943::__init__::232::IOProcess::(_processLogs) (981) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:27,943::__init__::232::IOProcess::(_processLogs) (981) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:27,944::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:27,944::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:27,944::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:27,944::__init__::232::IOProcess::(_processLogs) (982) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:27,944::__init__::232::IOProcess::(_processLogs) (982) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:28,678::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:28,679::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1147::DEBUG::2014-10-15 07:13:28,680::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b502dec-78f9-49fc-bc34-f63241ee1648`::moving from state init -> state preparing >Thread-1147::INFO::2014-10-15 07:13:28,680::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1147::INFO::2014-10-15 07:13:28,680::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000366136', 'lastCheck': '0.7', 'valid': True}} >Thread-1147::DEBUG::2014-10-15 07:13:28,680::task::1191::Storage.TaskManager.Task::(prepare) Task=`6b502dec-78f9-49fc-bc34-f63241ee1648`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000366136', 'lastCheck': '0.7', 'valid': True}} >Thread-1147::DEBUG::2014-10-15 07:13:28,681::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b502dec-78f9-49fc-bc34-f63241ee1648`::moving from state preparing -> state finished >Thread-1147::DEBUG::2014-10-15 07:13:28,681::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1147::DEBUG::2014-10-15 07:13:28,681::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1147::DEBUG::2014-10-15 07:13:28,681::task::993::Storage.TaskManager.Task::(_decref) Task=`6b502dec-78f9-49fc-bc34-f63241ee1648`::ref 0 aborting False >Thread-1147::ERROR::2014-10-15 07:13:28,682::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1147::DEBUG::2014-10-15 07:13:28,682::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:28,696::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:28,696::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1148::DEBUG::2014-10-15 07:13:28,697::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1148::DEBUG::2014-10-15 07:13:28,697::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:29,148::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:29,148::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:29,149::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:31,809::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:31,810::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1149::DEBUG::2014-10-15 07:13:31,813::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:31,815::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:31,816::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1150::DEBUG::2014-10-15 07:13:31,818::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1150::DEBUG::2014-10-15 07:13:31,819::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:34,154::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:34,155::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:34,155::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:34,840::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:34,840::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1151::DEBUG::2014-10-15 07:13:34,841::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:34,842::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:34,843::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1152::DEBUG::2014-10-15 07:13:34,844::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1152::DEBUG::2014-10-15 07:13:34,844::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:37,863::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:37,864::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1153::DEBUG::2014-10-15 07:13:37,865::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:37,866::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:37,867::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1154::DEBUG::2014-10-15 07:13:37,868::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1154::DEBUG::2014-10-15 07:13:37,868::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:13:37,945::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:37,945::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:37,945::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:37,946::__init__::232::IOProcess::(_processLogs) (983) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:13:37,946::__init__::232::IOProcess::(_processLogs) (983) Queuing response >Thread-25::DEBUG::2014-10-15 07:13:37,946::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:13:37,952::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000449908 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:13:37,953::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:37,953::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:37,953::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:37,953::__init__::232::IOProcess::(_processLogs) (984) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:13:37,954::__init__::232::IOProcess::(_processLogs) (984) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:37,954::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:37,954::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:37,954::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:37,954::__init__::232::IOProcess::(_processLogs) (985) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:37,954::__init__::232::IOProcess::(_processLogs) (985) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:37,955::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:37,955::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:37,955::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:37,955::__init__::232::IOProcess::(_processLogs) (986) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:37,955::__init__::232::IOProcess::(_processLogs) (986) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:37,956::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:37,956::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:37,956::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:37,956::__init__::232::IOProcess::(_processLogs) (987) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:37,956::__init__::232::IOProcess::(_processLogs) (987) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:39,161::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:39,162::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:39,162::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:40,890::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:40,890::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1155::DEBUG::2014-10-15 07:13:40,891::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:40,892::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:40,893::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1156::DEBUG::2014-10-15 07:13:40,894::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1156::DEBUG::2014-10-15 07:13:40,894::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:43,925::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:43,926::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1157::DEBUG::2014-10-15 07:13:43,927::task::595::Storage.TaskManager.Task::(_updateState) Task=`56503bdd-b327-42a5-a800-b89df60d313c`::moving from state init -> state preparing >Thread-1157::INFO::2014-10-15 07:13:43,927::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1157::INFO::2014-10-15 07:13:43,928::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000449908', 'lastCheck': '6.0', 'valid': True}} >Thread-1157::DEBUG::2014-10-15 07:13:43,928::task::1191::Storage.TaskManager.Task::(prepare) Task=`56503bdd-b327-42a5-a800-b89df60d313c`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000449908', 'lastCheck': '6.0', 'valid': True}} >Thread-1157::DEBUG::2014-10-15 07:13:43,928::task::595::Storage.TaskManager.Task::(_updateState) Task=`56503bdd-b327-42a5-a800-b89df60d313c`::moving from state preparing -> state finished >Thread-1157::DEBUG::2014-10-15 07:13:43,928::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1157::DEBUG::2014-10-15 07:13:43,928::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1157::DEBUG::2014-10-15 07:13:43,928::task::993::Storage.TaskManager.Task::(_decref) Task=`56503bdd-b327-42a5-a800-b89df60d313c`::ref 0 aborting False >Thread-1157::ERROR::2014-10-15 07:13:43,929::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1157::DEBUG::2014-10-15 07:13:43,930::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:43,943::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:43,943::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1158::DEBUG::2014-10-15 07:13:43,944::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1158::DEBUG::2014-10-15 07:13:43,945::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:44,168::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:44,168::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:44,168::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:47,040::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:47,040::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1159::DEBUG::2014-10-15 07:13:47,041::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:47,042::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:47,043::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1160::DEBUG::2014-10-15 07:13:47,044::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1160::DEBUG::2014-10-15 07:13:47,044::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:13:47,957::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:47,957::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:47,957::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:47,957::__init__::232::IOProcess::(_processLogs) (988) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:13:47,957::__init__::232::IOProcess::(_processLogs) (988) Queuing response >Thread-25::DEBUG::2014-10-15 07:13:47,958::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:13:47,964::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00038271 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:13:47,964::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:47,965::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:47,965::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:47,965::__init__::232::IOProcess::(_processLogs) (989) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:13:47,965::__init__::232::IOProcess::(_processLogs) (989) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:47,966::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:47,966::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:47,966::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:47,966::__init__::232::IOProcess::(_processLogs) (990) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:47,966::__init__::232::IOProcess::(_processLogs) (990) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:47,966::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:47,967::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:47,967::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:47,967::__init__::232::IOProcess::(_processLogs) (991) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:47,967::__init__::232::IOProcess::(_processLogs) (991) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:47,967::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:47,967::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:47,967::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:47,967::__init__::232::IOProcess::(_processLogs) (992) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:47,968::__init__::232::IOProcess::(_processLogs) (992) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:49,171::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:49,172::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:49,172::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:50,099::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:50,100::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1161::DEBUG::2014-10-15 07:13:50,101::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:50,102::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:50,102::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1162::DEBUG::2014-10-15 07:13:50,103::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1162::DEBUG::2014-10-15 07:13:50,104::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:53,120::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:53,121::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1163::DEBUG::2014-10-15 07:13:53,122::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:53,123::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:53,123::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1164::DEBUG::2014-10-15 07:13:53,124::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1164::DEBUG::2014-10-15 07:13:53,125::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:54,178::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:54,178::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:54,179::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:56,149::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:56,149::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1165::DEBUG::2014-10-15 07:13:56,150::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:56,151::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:56,152::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1166::DEBUG::2014-10-15 07:13:56,153::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1166::DEBUG::2014-10-15 07:13:56,153::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:13:56,564::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >Thread-16::DEBUG::2014-10-15 07:13:57,969::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:57,969::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:57,969::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:57,969::__init__::232::IOProcess::(_processLogs) (993) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:13:57,969::__init__::232::IOProcess::(_processLogs) (993) Queuing response >Thread-25::DEBUG::2014-10-15 07:13:57,970::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:13:57,975::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000418822 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:13:57,976::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:57,976::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:57,976::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:57,977::__init__::232::IOProcess::(_processLogs) (994) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:13:57,977::__init__::232::IOProcess::(_processLogs) (994) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:57,977::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:57,977::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:57,977::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:57,977::__init__::232::IOProcess::(_processLogs) (995) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:57,978::__init__::232::IOProcess::(_processLogs) (995) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:57,978::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:57,978::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:57,978::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:57,978::__init__::232::IOProcess::(_processLogs) (996) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:57,978::__init__::232::IOProcess::(_processLogs) (996) Queuing response >Thread-16::DEBUG::2014-10-15 07:13:57,979::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:13:57,979::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:13:57,979::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:13:57,979::__init__::232::IOProcess::(_processLogs) (997) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:13:57,979::__init__::232::IOProcess::(_processLogs) (997) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:59,176::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:59,177::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1167::DEBUG::2014-10-15 07:13:59,178::task::595::Storage.TaskManager.Task::(_updateState) Task=`e416d67c-43f4-4d20-ae10-886272c4cd05`::moving from state init -> state preparing >Thread-1167::INFO::2014-10-15 07:13:59,178::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1167::INFO::2014-10-15 07:13:59,178::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418822', 'lastCheck': '1.2', 'valid': True}} >Thread-1167::DEBUG::2014-10-15 07:13:59,179::task::1191::Storage.TaskManager.Task::(prepare) Task=`e416d67c-43f4-4d20-ae10-886272c4cd05`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418822', 'lastCheck': '1.2', 'valid': True}} >Thread-1167::DEBUG::2014-10-15 07:13:59,179::task::595::Storage.TaskManager.Task::(_updateState) Task=`e416d67c-43f4-4d20-ae10-886272c4cd05`::moving from state preparing -> state finished >Thread-1167::DEBUG::2014-10-15 07:13:59,179::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1167::DEBUG::2014-10-15 07:13:59,179::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1167::DEBUG::2014-10-15 07:13:59,179::task::993::Storage.TaskManager.Task::(_decref) Task=`e416d67c-43f4-4d20-ae10-886272c4cd05`::ref 0 aborting False >Thread-1167::ERROR::2014-10-15 07:13:59,180::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1167::DEBUG::2014-10-15 07:13:59,181::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:59,184::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:59,185::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:13:59,185::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:13:59,193::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:13:59,194::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1168::DEBUG::2014-10-15 07:13:59,195::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1168::DEBUG::2014-10-15 07:13:59,195::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:02,311::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:02,313::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1169::DEBUG::2014-10-15 07:14:02,315::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:02,318::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:02,319::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1170::DEBUG::2014-10-15 07:14:02,321::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1170::DEBUG::2014-10-15 07:14:02,322::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:04,191::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:04,191::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:04,192::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:05,381::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:05,382::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1171::DEBUG::2014-10-15 07:14:05,383::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:05,384::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:05,384::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1172::DEBUG::2014-10-15 07:14:05,385::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1172::DEBUG::2014-10-15 07:14:05,385::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:14:07,980::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:07,980::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:07,980::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:07,980::__init__::232::IOProcess::(_processLogs) (998) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:14:07,981::__init__::232::IOProcess::(_processLogs) (998) Queuing response >Thread-25::DEBUG::2014-10-15 07:14:07,981::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:14:07,989::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000464601 s, 988 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:14:07,989::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:07,989::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:07,989::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:07,990::__init__::232::IOProcess::(_processLogs) (999) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:14:07,990::__init__::232::IOProcess::(_processLogs) (999) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:07,990::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:07,990::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:07,990::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:07,991::__init__::232::IOProcess::(_processLogs) (1000) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:07,991::__init__::232::IOProcess::(_processLogs) (1000) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:07,991::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:07,991::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:07,991::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:07,991::__init__::232::IOProcess::(_processLogs) (1001) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:07,991::__init__::232::IOProcess::(_processLogs) (1001) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:07,992::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:07,992::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:07,992::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:07,992::__init__::232::IOProcess::(_processLogs) (1002) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:07,992::__init__::232::IOProcess::(_processLogs) (1002) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:08,428::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:08,429::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1173::DEBUG::2014-10-15 07:14:08,429::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:08,431::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:08,431::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1174::DEBUG::2014-10-15 07:14:08,432::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1174::DEBUG::2014-10-15 07:14:08,433::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:09,198::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:09,198::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:09,199::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:11,447::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:11,448::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1175::DEBUG::2014-10-15 07:14:11,449::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:11,450::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:11,451::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1176::DEBUG::2014-10-15 07:14:11,452::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1176::DEBUG::2014-10-15 07:14:11,452::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:14,204::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:14,205::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:14,205::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:14,463::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:14,463::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1177::DEBUG::2014-10-15 07:14:14,464::task::595::Storage.TaskManager.Task::(_updateState) Task=`55869f93-7223-4920-806c-c2f765ec3883`::moving from state init -> state preparing >Thread-1177::INFO::2014-10-15 07:14:14,464::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1177::INFO::2014-10-15 07:14:14,465::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000464601', 'lastCheck': '6.5', 'valid': True}} >Thread-1177::DEBUG::2014-10-15 07:14:14,465::task::1191::Storage.TaskManager.Task::(prepare) Task=`55869f93-7223-4920-806c-c2f765ec3883`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000464601', 'lastCheck': '6.5', 'valid': True}} >Thread-1177::DEBUG::2014-10-15 07:14:14,465::task::595::Storage.TaskManager.Task::(_updateState) Task=`55869f93-7223-4920-806c-c2f765ec3883`::moving from state preparing -> state finished >Thread-1177::DEBUG::2014-10-15 07:14:14,465::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1177::DEBUG::2014-10-15 07:14:14,465::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1177::DEBUG::2014-10-15 07:14:14,465::task::993::Storage.TaskManager.Task::(_decref) Task=`55869f93-7223-4920-806c-c2f765ec3883`::ref 0 aborting False >Thread-1177::ERROR::2014-10-15 07:14:14,466::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1177::DEBUG::2014-10-15 07:14:14,467::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:14,476::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:14,476::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1178::DEBUG::2014-10-15 07:14:14,477::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1178::DEBUG::2014-10-15 07:14:14,478::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:17,589::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:17,590::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1179::DEBUG::2014-10-15 07:14:17,591::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:17,592::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:17,592::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1180::DEBUG::2014-10-15 07:14:17,593::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1180::DEBUG::2014-10-15 07:14:17,594::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:14:17,993::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:17,993::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:17,994::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:17,994::__init__::232::IOProcess::(_processLogs) (1003) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:14:17,994::__init__::232::IOProcess::(_processLogs) (1003) Queuing response >Thread-25::DEBUG::2014-10-15 07:14:17,994::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:14:18,000::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000362797 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:14:18,001::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:18,001::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:18,001::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:18,001::__init__::232::IOProcess::(_processLogs) (1004) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:14:18,002::__init__::232::IOProcess::(_processLogs) (1004) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:18,002::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:18,002::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:18,002::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:18,002::__init__::232::IOProcess::(_processLogs) (1005) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:18,002::__init__::232::IOProcess::(_processLogs) (1005) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:18,003::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:18,003::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:18,003::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:18,003::__init__::232::IOProcess::(_processLogs) (1006) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:18,003::__init__::232::IOProcess::(_processLogs) (1006) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:18,004::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:18,004::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:18,004::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:18,004::__init__::232::IOProcess::(_processLogs) (1007) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:18,004::__init__::232::IOProcess::(_processLogs) (1007) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:19,211::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:19,211::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:19,212::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:20,658::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:20,659::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1181::DEBUG::2014-10-15 07:14:20,660::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:20,661::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:20,662::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1182::DEBUG::2014-10-15 07:14:20,663::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1182::DEBUG::2014-10-15 07:14:20,664::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:23,682::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:23,682::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1183::DEBUG::2014-10-15 07:14:23,683::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:23,685::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:23,685::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1184::DEBUG::2014-10-15 07:14:23,686::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1184::DEBUG::2014-10-15 07:14:23,686::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:24,217::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:24,218::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:24,218::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1065::DEBUG::2014-10-15 07:14:26,250::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3df0e87-4ca9-4f60-81d6-19995f7f0992`::moving from state init -> state preparing >Thread-1065::INFO::2014-10-15 07:14:26,250::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', volUUID='486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:14:26,250::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:26,251::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:26,251::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:26,251::__init__::232::IOProcess::(_processLogs) (1008) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:14:26,251::__init__::232::IOProcess::(_processLogs) (1008) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:26,252::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:26,252::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:26,252::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:26,252::__init__::232::IOProcess::(_processLogs) (1009) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:14:26,252::__init__::232::IOProcess::(_processLogs) (1009) Queuing response >Thread-1065::INFO::2014-10-15 07:14:26,253::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:14:26,253::task::1191::Storage.TaskManager.Task::(prepare) Task=`a3df0e87-4ca9-4f60-81d6-19995f7f0992`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:14:26,253::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3df0e87-4ca9-4f60-81d6-19995f7f0992`::moving from state preparing -> state finished >Thread-1065::DEBUG::2014-10-15 07:14:26,253::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1065::DEBUG::2014-10-15 07:14:26,253::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1065::DEBUG::2014-10-15 07:14:26,253::task::993::Storage.TaskManager.Task::(_decref) Task=`a3df0e87-4ca9-4f60-81d6-19995f7f0992`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 07:14:26,597::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:26,708::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:26,709::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1185::DEBUG::2014-10-15 07:14:26,710::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:26,715::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:26,715::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1186::DEBUG::2014-10-15 07:14:26,716::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1186::DEBUG::2014-10-15 07:14:26,716::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-25::DEBUG::2014-10-15 07:14:28,005::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 07:14:28,006::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:28,006::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:28,006::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:28,006::__init__::232::IOProcess::(_processLogs) (1010) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:14:28,006::__init__::232::IOProcess::(_processLogs) (1010) Queuing response >Thread-25::DEBUG::2014-10-15 07:14:28,013::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 07:14:28,013::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:28,014::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:28,014::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:28,014::__init__::232::IOProcess::(_processLogs) (1011) Got request for method 'touch' >Thread-16::DEBUG::2014-10-15 07:14:28,055::__init__::232::IOProcess::(_processLogs) (1011) Queuing response >Thread-25::DEBUG::2014-10-15 07:14:28,056::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-16::DEBUG::2014-10-15 07:14:28,056::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:28,056::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:28,056::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:28,056::__init__::232::IOProcess::(_processLogs) (1012) Got request for method 'readfile' >Thread-16::DEBUG::2014-10-15 07:14:28,057::__init__::232::IOProcess::(_processLogs) (1012) Queuing response >Thread-25::DEBUG::2014-10-15 07:14:28,057::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=domain1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-0000000003e1', 'REMOTE_PATH=10.66.128.104:/home/NFS1', 'ROLE=Master', 'SDUUID=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b90f33781d11f6e6b0b3945de3167c0719a458c1'] >Thread-16::DEBUG::2014-10-15 07:14:28,058::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:28,058::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:28,058::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:28,058::__init__::232::IOProcess::(_processLogs) (1013) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:28,058::__init__::232::IOProcess::(_processLogs) (1013) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:28,059::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:28,059::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:28,059::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:28,059::__init__::232::IOProcess::(_processLogs) (1014) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:14:28,059::__init__::232::IOProcess::(_processLogs) (1014) Queuing response >Thread-25::DEBUG::2014-10-15 07:14:28,059::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-25::INFO::2014-10-15 07:14:28,059::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_imageNS already registered >Thread-25::INFO::2014-10-15 07:14:28,059::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_volumeNS already registered >Thread-16::DEBUG::2014-10-15 07:14:28,060::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:28,060::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:28,060::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:28,060::__init__::232::IOProcess::(_processLogs) (1015) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:14:28,060::__init__::232::IOProcess::(_processLogs) (1015) Queuing response >Thread-25::DEBUG::2014-10-15 07:14:28,060::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:14:28,067::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000371094 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:14:28,067::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:28,067::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:28,068::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:28,068::__init__::232::IOProcess::(_processLogs) (1016) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:14:28,068::__init__::232::IOProcess::(_processLogs) (1016) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:28,068::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:28,068::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:28,069::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:28,069::__init__::232::IOProcess::(_processLogs) (1017) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:28,069::__init__::232::IOProcess::(_processLogs) (1017) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:28,069::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:28,069::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:28,069::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:28,069::__init__::232::IOProcess::(_processLogs) (1018) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:28,069::__init__::232::IOProcess::(_processLogs) (1018) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:28,070::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:28,070::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:28,070::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:28,070::__init__::232::IOProcess::(_processLogs) (1019) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:28,070::__init__::232::IOProcess::(_processLogs) (1019) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:29,222::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:29,222::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:29,223::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:29,741::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:29,742::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1188::DEBUG::2014-10-15 07:14:29,743::task::595::Storage.TaskManager.Task::(_updateState) Task=`caa16671-847a-4feb-8ae0-ed7f136ac616`::moving from state init -> state preparing >Thread-1188::INFO::2014-10-15 07:14:29,743::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1188::INFO::2014-10-15 07:14:29,743::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000371094', 'lastCheck': '1.7', 'valid': True}} >Thread-1188::DEBUG::2014-10-15 07:14:29,743::task::1191::Storage.TaskManager.Task::(prepare) Task=`caa16671-847a-4feb-8ae0-ed7f136ac616`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000371094', 'lastCheck': '1.7', 'valid': True}} >Thread-1188::DEBUG::2014-10-15 07:14:29,744::task::595::Storage.TaskManager.Task::(_updateState) Task=`caa16671-847a-4feb-8ae0-ed7f136ac616`::moving from state preparing -> state finished >Thread-1188::DEBUG::2014-10-15 07:14:29,744::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1188::DEBUG::2014-10-15 07:14:29,744::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1188::DEBUG::2014-10-15 07:14:29,744::task::993::Storage.TaskManager.Task::(_decref) Task=`caa16671-847a-4feb-8ae0-ed7f136ac616`::ref 0 aborting False >Thread-1188::ERROR::2014-10-15 07:14:29,745::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1188::DEBUG::2014-10-15 07:14:29,745::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:29,755::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:29,756::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1189::DEBUG::2014-10-15 07:14:29,757::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1189::DEBUG::2014-10-15 07:14:29,757::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:32,868::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:32,868::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1190::DEBUG::2014-10-15 07:14:32,870::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:32,871::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:32,871::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1191::DEBUG::2014-10-15 07:14:32,872::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1191::DEBUG::2014-10-15 07:14:32,873::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:34,229::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:34,229::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:34,229::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:35,904::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:35,905::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1192::DEBUG::2014-10-15 07:14:35,906::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:35,907::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:35,907::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1193::DEBUG::2014-10-15 07:14:35,908::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1193::DEBUG::2014-10-15 07:14:35,908::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:14:38,071::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:38,071::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:38,071::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:38,072::__init__::232::IOProcess::(_processLogs) (1020) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:14:38,072::__init__::232::IOProcess::(_processLogs) (1020) Queuing response >Thread-25::DEBUG::2014-10-15 07:14:38,072::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:14:38,078::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000461075 s, 995 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:14:38,079::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:38,079::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:38,079::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:38,079::__init__::232::IOProcess::(_processLogs) (1021) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:14:38,080::__init__::232::IOProcess::(_processLogs) (1021) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:38,080::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:38,080::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:38,080::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:38,080::__init__::232::IOProcess::(_processLogs) (1022) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:38,080::__init__::232::IOProcess::(_processLogs) (1022) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:38,081::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:38,081::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:38,081::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:38,081::__init__::232::IOProcess::(_processLogs) (1023) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:38,081::__init__::232::IOProcess::(_processLogs) (1023) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:38,082::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:38,082::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:38,082::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:38,082::__init__::232::IOProcess::(_processLogs) (1024) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:38,082::__init__::232::IOProcess::(_processLogs) (1024) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:38,924::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:38,925::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1194::DEBUG::2014-10-15 07:14:38,926::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:38,927::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:38,927::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1195::DEBUG::2014-10-15 07:14:38,928::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1195::DEBUG::2014-10-15 07:14:38,929::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:39,234::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:39,235::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:39,235::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:41,943::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:41,944::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1196::DEBUG::2014-10-15 07:14:41,945::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:41,946::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:41,947::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1197::DEBUG::2014-10-15 07:14:41,948::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1197::DEBUG::2014-10-15 07:14:41,948::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:44,241::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:44,241::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:44,242::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:44,964::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:44,965::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1198::DEBUG::2014-10-15 07:14:44,966::task::595::Storage.TaskManager.Task::(_updateState) Task=`11342198-96fd-446d-a8b3-d070e02138b6`::moving from state init -> state preparing >Thread-1198::INFO::2014-10-15 07:14:44,966::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1198::INFO::2014-10-15 07:14:44,966::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000461075', 'lastCheck': '6.9', 'valid': True}} >Thread-1198::DEBUG::2014-10-15 07:14:44,966::task::1191::Storage.TaskManager.Task::(prepare) Task=`11342198-96fd-446d-a8b3-d070e02138b6`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000461075', 'lastCheck': '6.9', 'valid': True}} >Thread-1198::DEBUG::2014-10-15 07:14:44,966::task::595::Storage.TaskManager.Task::(_updateState) Task=`11342198-96fd-446d-a8b3-d070e02138b6`::moving from state preparing -> state finished >Thread-1198::DEBUG::2014-10-15 07:14:44,966::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1198::DEBUG::2014-10-15 07:14:44,966::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1198::DEBUG::2014-10-15 07:14:44,967::task::993::Storage.TaskManager.Task::(_decref) Task=`11342198-96fd-446d-a8b3-d070e02138b6`::ref 0 aborting False >Thread-1198::ERROR::2014-10-15 07:14:44,967::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1198::DEBUG::2014-10-15 07:14:44,968::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:44,977::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:44,977::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1199::DEBUG::2014-10-15 07:14:44,978::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1199::DEBUG::2014-10-15 07:14:44,978::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:14:48,083::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:48,083::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:48,083::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:48,083::__init__::232::IOProcess::(_processLogs) (1025) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:14:48,083::__init__::232::IOProcess::(_processLogs) (1025) Queuing response >Thread-25::DEBUG::2014-10-15 07:14:48,084::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:14:48,090::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000394322 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:14:48,091::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:48,091::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:48,091::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:48,091::__init__::232::IOProcess::(_processLogs) (1026) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:14:48,091::__init__::232::IOProcess::(_processLogs) (1026) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:48,092::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-16::DEBUG::2014-10-15 07:14:48,093::__init__::232::IOProcess::(_processLogs) Receiving request... >JsonRpcServer::DEBUG::2014-10-15 07:14:48,094::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-16::DEBUG::2014-10-15 07:14:48,094::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:48,095::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:48,096::__init__::232::IOProcess::(_processLogs) (1027) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:48,097::__init__::232::IOProcess::(_processLogs) (1027) Queuing response >Thread-1200::DEBUG::2014-10-15 07:14:48,097::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:14:48,098::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:48,099::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:48,099::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:48,099::__init__::232::IOProcess::(_processLogs) (1028) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:48,099::__init__::232::IOProcess::(_processLogs) (1028) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:48,100::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-16::DEBUG::2014-10-15 07:14:48,100::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:48,100::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >JsonRpcServer::DEBUG::2014-10-15 07:14:48,101::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-16::DEBUG::2014-10-15 07:14:48,102::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:48,102::__init__::232::IOProcess::(_processLogs) (1029) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:48,103::__init__::232::IOProcess::(_processLogs) (1029) Queuing response >Thread-1201::DEBUG::2014-10-15 07:14:48,103::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1201::DEBUG::2014-10-15 07:14:48,104::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:49,247::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:49,248::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:49,248::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:51,274::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:51,275::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1202::DEBUG::2014-10-15 07:14:51,276::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:51,277::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:51,277::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1203::DEBUG::2014-10-15 07:14:51,278::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1203::DEBUG::2014-10-15 07:14:51,278::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:54,254::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:54,255::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:54,255::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:54,297::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:54,298::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1204::DEBUG::2014-10-15 07:14:54,299::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:54,300::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:54,300::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1205::DEBUG::2014-10-15 07:14:54,301::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1205::DEBUG::2014-10-15 07:14:54,301::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:14:56,630::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:57,321::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:57,322::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1206::DEBUG::2014-10-15 07:14:57,323::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:14:57,324::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:14:57,324::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1207::DEBUG::2014-10-15 07:14:57,325::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1207::DEBUG::2014-10-15 07:14:57,326::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:14:58,105::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:58,105::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:58,105::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:58,106::__init__::232::IOProcess::(_processLogs) (1030) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:14:58,106::__init__::232::IOProcess::(_processLogs) (1030) Queuing response >Thread-25::DEBUG::2014-10-15 07:14:58,106::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:14:58,112::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000392193 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:14:58,113::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:58,113::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:58,113::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:58,113::__init__::232::IOProcess::(_processLogs) (1031) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:14:58,113::__init__::232::IOProcess::(_processLogs) (1031) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:58,113::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:58,114::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:58,114::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:58,114::__init__::232::IOProcess::(_processLogs) (1032) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:58,114::__init__::232::IOProcess::(_processLogs) (1032) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:58,114::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:58,114::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:58,114::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:58,114::__init__::232::IOProcess::(_processLogs) (1033) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:58,115::__init__::232::IOProcess::(_processLogs) (1033) Queuing response >Thread-16::DEBUG::2014-10-15 07:14:58,115::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:14:58,115::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:14:58,115::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:14:58,115::__init__::232::IOProcess::(_processLogs) (1034) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:14:58,115::__init__::232::IOProcess::(_processLogs) (1034) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:59,261::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:59,262::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:14:59,262::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:00,350::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:00,351::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1208::DEBUG::2014-10-15 07:15:00,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb02e1d1-be85-42b9-adcd-01efb974f555`::moving from state init -> state preparing >Thread-1208::INFO::2014-10-15 07:15:00,352::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1208::INFO::2014-10-15 07:15:00,353::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000392193', 'lastCheck': '2.2', 'valid': True}} >Thread-1208::DEBUG::2014-10-15 07:15:00,353::task::1191::Storage.TaskManager.Task::(prepare) Task=`bb02e1d1-be85-42b9-adcd-01efb974f555`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000392193', 'lastCheck': '2.2', 'valid': True}} >Thread-1208::DEBUG::2014-10-15 07:15:00,353::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb02e1d1-be85-42b9-adcd-01efb974f555`::moving from state preparing -> state finished >Thread-1208::DEBUG::2014-10-15 07:15:00,353::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1208::DEBUG::2014-10-15 07:15:00,353::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1208::DEBUG::2014-10-15 07:15:00,353::task::993::Storage.TaskManager.Task::(_decref) Task=`bb02e1d1-be85-42b9-adcd-01efb974f555`::ref 0 aborting False >Thread-1208::ERROR::2014-10-15 07:15:00,354::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1208::DEBUG::2014-10-15 07:15:00,355::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:00,366::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:00,366::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1209::DEBUG::2014-10-15 07:15:00,367::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1209::DEBUG::2014-10-15 07:15:00,368::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:03,471::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:03,472::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1210::DEBUG::2014-10-15 07:15:03,473::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:03,474::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:03,474::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1211::DEBUG::2014-10-15 07:15:03,475::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1211::DEBUG::2014-10-15 07:15:03,476::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:04,268::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:04,268::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:04,269::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:06,513::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:06,514::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1212::DEBUG::2014-10-15 07:15:06,515::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:06,516::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:06,516::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1213::DEBUG::2014-10-15 07:15:06,517::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1213::DEBUG::2014-10-15 07:15:06,518::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:15:08,116::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:08,116::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:08,117::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:08,117::__init__::232::IOProcess::(_processLogs) (1035) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:15:08,117::__init__::232::IOProcess::(_processLogs) (1035) Queuing response >Thread-25::DEBUG::2014-10-15 07:15:08,118::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:15:08,125::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000417862 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:15:08,125::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:08,125::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:08,126::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:08,126::__init__::232::IOProcess::(_processLogs) (1036) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:15:08,126::__init__::232::IOProcess::(_processLogs) (1036) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:08,126::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:08,127::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:08,127::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:08,127::__init__::232::IOProcess::(_processLogs) (1037) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:08,127::__init__::232::IOProcess::(_processLogs) (1037) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:08,127::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:08,127::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:08,127::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:08,128::__init__::232::IOProcess::(_processLogs) (1038) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:08,128::__init__::232::IOProcess::(_processLogs) (1038) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:08,128::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:08,128::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:08,128::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:08,128::__init__::232::IOProcess::(_processLogs) (1039) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:08,128::__init__::232::IOProcess::(_processLogs) (1039) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:09,274::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:09,275::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:09,275::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:09,534::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:09,535::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1214::DEBUG::2014-10-15 07:15:09,536::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:09,537::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:09,538::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1215::DEBUG::2014-10-15 07:15:09,538::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1215::DEBUG::2014-10-15 07:15:09,539::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:12,562::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:12,563::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1216::DEBUG::2014-10-15 07:15:12,564::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:12,565::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:12,565::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1217::DEBUG::2014-10-15 07:15:12,566::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1217::DEBUG::2014-10-15 07:15:12,567::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:14,281::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:14,281::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:14,282::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:15,583::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:15,584::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1218::DEBUG::2014-10-15 07:15:15,585::task::595::Storage.TaskManager.Task::(_updateState) Task=`3dad54af-f2e6-4090-ae00-43b4484ddad7`::moving from state init -> state preparing >Thread-1218::INFO::2014-10-15 07:15:15,585::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1218::INFO::2014-10-15 07:15:15,586::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417862', 'lastCheck': '7.5', 'valid': True}} >Thread-1218::DEBUG::2014-10-15 07:15:15,586::task::1191::Storage.TaskManager.Task::(prepare) Task=`3dad54af-f2e6-4090-ae00-43b4484ddad7`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417862', 'lastCheck': '7.5', 'valid': True}} >Thread-1218::DEBUG::2014-10-15 07:15:15,586::task::595::Storage.TaskManager.Task::(_updateState) Task=`3dad54af-f2e6-4090-ae00-43b4484ddad7`::moving from state preparing -> state finished >Thread-1218::DEBUG::2014-10-15 07:15:15,586::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1218::DEBUG::2014-10-15 07:15:15,586::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1218::DEBUG::2014-10-15 07:15:15,586::task::993::Storage.TaskManager.Task::(_decref) Task=`3dad54af-f2e6-4090-ae00-43b4484ddad7`::ref 0 aborting False >Thread-1218::ERROR::2014-10-15 07:15:15,587::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1218::DEBUG::2014-10-15 07:15:15,588::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:15,598::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:15,599::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1219::DEBUG::2014-10-15 07:15:15,599::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1219::DEBUG::2014-10-15 07:15:15,600::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:15:18,129::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:18,130::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:18,130::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:18,130::__init__::232::IOProcess::(_processLogs) (1040) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:15:18,130::__init__::232::IOProcess::(_processLogs) (1040) Queuing response >Thread-25::DEBUG::2014-10-15 07:15:18,130::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:15:18,137::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000399466 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:15:18,137::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:18,137::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:18,138::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:18,138::__init__::232::IOProcess::(_processLogs) (1041) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:15:18,138::__init__::232::IOProcess::(_processLogs) (1041) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:18,138::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:18,138::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:18,139::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:18,139::__init__::232::IOProcess::(_processLogs) (1042) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:18,139::__init__::232::IOProcess::(_processLogs) (1042) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:18,139::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:18,139::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:18,139::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:18,139::__init__::232::IOProcess::(_processLogs) (1043) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:18,139::__init__::232::IOProcess::(_processLogs) (1043) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:18,140::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:18,140::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:18,140::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:18,140::__init__::232::IOProcess::(_processLogs) (1044) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:18,140::__init__::232::IOProcess::(_processLogs) (1044) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:18,697::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:18,698::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1220::DEBUG::2014-10-15 07:15:18,699::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:18,700::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:18,701::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1221::DEBUG::2014-10-15 07:15:18,702::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1221::DEBUG::2014-10-15 07:15:18,702::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:19,288::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:19,288::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:19,289::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:21,766::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:21,767::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1222::DEBUG::2014-10-15 07:15:21,768::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:21,769::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:21,770::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1223::DEBUG::2014-10-15 07:15:21,770::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1223::DEBUG::2014-10-15 07:15:21,771::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:24,294::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:24,295::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:24,295::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:24,799::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:24,799::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1224::DEBUG::2014-10-15 07:15:24,800::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:24,801::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:24,802::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1225::DEBUG::2014-10-15 07:15:24,803::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1225::DEBUG::2014-10-15 07:15:24,803::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-1065::DEBUG::2014-10-15 07:15:26,292::task::595::Storage.TaskManager.Task::(_updateState) Task=`73f30614-53b8-4342-b8b7-c2c51ff9b4f3`::moving from state init -> state preparing >Thread-1065::INFO::2014-10-15 07:15:26,292::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', volUUID='486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:15:26,293::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:26,293::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:26,293::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:26,293::__init__::232::IOProcess::(_processLogs) (1045) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:15:26,294::__init__::232::IOProcess::(_processLogs) (1045) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:26,294::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:26,294::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:26,294::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:26,294::__init__::232::IOProcess::(_processLogs) (1046) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:15:26,295::__init__::232::IOProcess::(_processLogs) (1046) Queuing response >Thread-1065::INFO::2014-10-15 07:15:26,295::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:15:26,295::task::1191::Storage.TaskManager.Task::(prepare) Task=`73f30614-53b8-4342-b8b7-c2c51ff9b4f3`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:15:26,295::task::595::Storage.TaskManager.Task::(_updateState) Task=`73f30614-53b8-4342-b8b7-c2c51ff9b4f3`::moving from state preparing -> state finished >Thread-1065::DEBUG::2014-10-15 07:15:26,295::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1065::DEBUG::2014-10-15 07:15:26,295::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1065::DEBUG::2014-10-15 07:15:26,295::task::993::Storage.TaskManager.Task::(_decref) Task=`73f30614-53b8-4342-b8b7-c2c51ff9b4f3`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 07:15:26,663::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:27,821::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:27,822::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1226::DEBUG::2014-10-15 07:15:27,823::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:27,824::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:27,825::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1227::DEBUG::2014-10-15 07:15:27,825::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1227::DEBUG::2014-10-15 07:15:27,826::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:15:28,141::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:28,141::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:28,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:28,142::__init__::232::IOProcess::(_processLogs) (1047) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:15:28,142::__init__::232::IOProcess::(_processLogs) (1047) Queuing response >Thread-25::DEBUG::2014-10-15 07:15:28,142::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:15:28,148::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000430325 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:15:28,149::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:28,149::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:28,149::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:28,149::__init__::232::IOProcess::(_processLogs) (1048) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:15:28,150::__init__::232::IOProcess::(_processLogs) (1048) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:28,150::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:28,150::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:28,150::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:28,150::__init__::232::IOProcess::(_processLogs) (1049) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:28,151::__init__::232::IOProcess::(_processLogs) (1049) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:28,151::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:28,151::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:28,151::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:28,151::__init__::232::IOProcess::(_processLogs) (1050) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:28,151::__init__::232::IOProcess::(_processLogs) (1050) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:28,152::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:28,152::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:28,152::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:28,152::__init__::232::IOProcess::(_processLogs) (1051) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:28,152::__init__::232::IOProcess::(_processLogs) (1051) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:29,301::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:29,301::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:29,302::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:30,839::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:30,840::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1228::DEBUG::2014-10-15 07:15:30,841::task::595::Storage.TaskManager.Task::(_updateState) Task=`0be71c33-3d16-474d-a47f-d173673f6c1f`::moving from state init -> state preparing >Thread-1228::INFO::2014-10-15 07:15:30,841::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1228::INFO::2014-10-15 07:15:30,841::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000430325', 'lastCheck': '2.7', 'valid': True}} >Thread-1228::DEBUG::2014-10-15 07:15:30,841::task::1191::Storage.TaskManager.Task::(prepare) Task=`0be71c33-3d16-474d-a47f-d173673f6c1f`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000430325', 'lastCheck': '2.7', 'valid': True}} >Thread-1228::DEBUG::2014-10-15 07:15:30,842::task::595::Storage.TaskManager.Task::(_updateState) Task=`0be71c33-3d16-474d-a47f-d173673f6c1f`::moving from state preparing -> state finished >Thread-1228::DEBUG::2014-10-15 07:15:30,842::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1228::DEBUG::2014-10-15 07:15:30,842::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1228::DEBUG::2014-10-15 07:15:30,842::task::993::Storage.TaskManager.Task::(_decref) Task=`0be71c33-3d16-474d-a47f-d173673f6c1f`::ref 0 aborting False >Thread-1228::ERROR::2014-10-15 07:15:30,843::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1228::DEBUG::2014-10-15 07:15:30,843::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:30,853::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:30,854::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1229::DEBUG::2014-10-15 07:15:30,855::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1229::DEBUG::2014-10-15 07:15:30,855::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:33,962::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:33,963::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1230::DEBUG::2014-10-15 07:15:33,964::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:33,965::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:33,966::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1231::DEBUG::2014-10-15 07:15:33,967::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1231::DEBUG::2014-10-15 07:15:33,967::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:34,307::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:34,308::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:34,308::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:36,994::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:36,995::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1232::DEBUG::2014-10-15 07:15:36,996::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:36,997::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:36,998::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1233::DEBUG::2014-10-15 07:15:36,999::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1233::DEBUG::2014-10-15 07:15:36,999::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:15:38,153::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:38,153::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:38,153::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:38,153::__init__::232::IOProcess::(_processLogs) (1052) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:15:38,154::__init__::232::IOProcess::(_processLogs) (1052) Queuing response >Thread-25::DEBUG::2014-10-15 07:15:38,154::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:15:38,160::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000447529 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:15:38,161::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:38,161::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:38,161::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:38,161::__init__::232::IOProcess::(_processLogs) (1053) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:15:38,161::__init__::232::IOProcess::(_processLogs) (1053) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:38,162::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:38,162::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:38,162::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:38,162::__init__::232::IOProcess::(_processLogs) (1054) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:38,162::__init__::232::IOProcess::(_processLogs) (1054) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:38,163::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:38,163::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:38,163::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:38,163::__init__::232::IOProcess::(_processLogs) (1055) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:38,163::__init__::232::IOProcess::(_processLogs) (1055) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:38,164::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:38,164::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:38,164::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:38,164::__init__::232::IOProcess::(_processLogs) (1056) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:38,164::__init__::232::IOProcess::(_processLogs) (1056) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:39,314::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:39,315::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:39,315::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:40,020::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:40,020::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1234::DEBUG::2014-10-15 07:15:40,021::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:40,023::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:40,023::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1235::DEBUG::2014-10-15 07:15:40,024::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1235::DEBUG::2014-10-15 07:15:40,024::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:43,050::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:43,051::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1236::DEBUG::2014-10-15 07:15:43,052::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:43,053::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:43,054::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1237::DEBUG::2014-10-15 07:15:43,054::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1237::DEBUG::2014-10-15 07:15:43,055::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:44,321::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:44,321::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:44,322::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:46,070::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:46,071::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1238::DEBUG::2014-10-15 07:15:46,072::task::595::Storage.TaskManager.Task::(_updateState) Task=`fbe701cf-fd4e-442e-9027-9ce8de196549`::moving from state init -> state preparing >Thread-1238::INFO::2014-10-15 07:15:46,072::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1238::INFO::2014-10-15 07:15:46,072::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000447529', 'lastCheck': '7.9', 'valid': True}} >Thread-1238::DEBUG::2014-10-15 07:15:46,072::task::1191::Storage.TaskManager.Task::(prepare) Task=`fbe701cf-fd4e-442e-9027-9ce8de196549`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000447529', 'lastCheck': '7.9', 'valid': True}} >Thread-1238::DEBUG::2014-10-15 07:15:46,072::task::595::Storage.TaskManager.Task::(_updateState) Task=`fbe701cf-fd4e-442e-9027-9ce8de196549`::moving from state preparing -> state finished >Thread-1238::DEBUG::2014-10-15 07:15:46,072::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1238::DEBUG::2014-10-15 07:15:46,072::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1238::DEBUG::2014-10-15 07:15:46,073::task::993::Storage.TaskManager.Task::(_decref) Task=`fbe701cf-fd4e-442e-9027-9ce8de196549`::ref 0 aborting False >Thread-1238::ERROR::2014-10-15 07:15:46,073::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1238::DEBUG::2014-10-15 07:15:46,074::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:46,083::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:46,083::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1239::DEBUG::2014-10-15 07:15:46,084::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1239::DEBUG::2014-10-15 07:15:46,084::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:15:48,165::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:48,165::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:48,165::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:48,165::__init__::232::IOProcess::(_processLogs) (1057) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:15:48,166::__init__::232::IOProcess::(_processLogs) (1057) Queuing response >Thread-25::DEBUG::2014-10-15 07:15:48,166::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:15:48,172::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000427077 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:15:48,173::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:48,173::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:48,173::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:48,173::__init__::232::IOProcess::(_processLogs) (1058) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:15:48,173::__init__::232::IOProcess::(_processLogs) (1058) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:48,174::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:48,174::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:48,174::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:48,174::__init__::232::IOProcess::(_processLogs) (1059) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:48,174::__init__::232::IOProcess::(_processLogs) (1059) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:48,175::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:48,175::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:48,175::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:48,175::__init__::232::IOProcess::(_processLogs) (1060) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:48,175::__init__::232::IOProcess::(_processLogs) (1060) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:48,176::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:48,176::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:48,176::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:48,176::__init__::232::IOProcess::(_processLogs) (1061) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:48,176::__init__::232::IOProcess::(_processLogs) (1061) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:49,196::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:49,196::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1240::DEBUG::2014-10-15 07:15:49,197::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:49,199::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:49,199::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1241::DEBUG::2014-10-15 07:15:49,200::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1241::DEBUG::2014-10-15 07:15:49,200::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:49,327::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:49,328::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:49,328::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:52,238::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:52,239::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1242::DEBUG::2014-10-15 07:15:52,240::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:52,241::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:52,241::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1243::DEBUG::2014-10-15 07:15:52,242::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1243::DEBUG::2014-10-15 07:15:52,243::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:54,334::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:54,335::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:54,335::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:55,260::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:55,260::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1244::DEBUG::2014-10-15 07:15:55,261::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:55,262::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:55,263::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1245::DEBUG::2014-10-15 07:15:55,264::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1245::DEBUG::2014-10-15 07:15:55,264::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:15:56,694::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >Thread-16::DEBUG::2014-10-15 07:15:58,177::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:58,177::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:58,177::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:58,177::__init__::232::IOProcess::(_processLogs) (1062) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:15:58,178::__init__::232::IOProcess::(_processLogs) (1062) Queuing response >Thread-25::DEBUG::2014-10-15 07:15:58,178::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:15:58,184::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000382304 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:15:58,185::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:58,185::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:58,185::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:58,185::__init__::232::IOProcess::(_processLogs) (1063) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:15:58,185::__init__::232::IOProcess::(_processLogs) (1063) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:58,186::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:58,186::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:58,186::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:58,186::__init__::232::IOProcess::(_processLogs) (1064) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:58,186::__init__::232::IOProcess::(_processLogs) (1064) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:58,187::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:58,187::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:58,187::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:58,187::__init__::232::IOProcess::(_processLogs) (1065) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:58,187::__init__::232::IOProcess::(_processLogs) (1065) Queuing response >Thread-16::DEBUG::2014-10-15 07:15:58,187::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:15:58,188::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:15:58,188::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:15:58,188::__init__::232::IOProcess::(_processLogs) (1066) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:15:58,188::__init__::232::IOProcess::(_processLogs) (1066) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:58,306::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:58,307::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1246::DEBUG::2014-10-15 07:15:58,308::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:15:58,309::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:15:58,310::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1247::DEBUG::2014-10-15 07:15:58,311::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1247::DEBUG::2014-10-15 07:15:58,311::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:59,341::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:59,341::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:15:59,342::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:01,334::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:01,335::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1248::DEBUG::2014-10-15 07:16:01,336::task::595::Storage.TaskManager.Task::(_updateState) Task=`7f93e2fd-2717-45ab-8e1a-e6e794606a21`::moving from state init -> state preparing >Thread-1248::INFO::2014-10-15 07:16:01,336::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1248::INFO::2014-10-15 07:16:01,336::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000382304', 'lastCheck': '3.1', 'valid': True}} >Thread-1248::DEBUG::2014-10-15 07:16:01,336::task::1191::Storage.TaskManager.Task::(prepare) Task=`7f93e2fd-2717-45ab-8e1a-e6e794606a21`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000382304', 'lastCheck': '3.1', 'valid': True}} >Thread-1248::DEBUG::2014-10-15 07:16:01,336::task::595::Storage.TaskManager.Task::(_updateState) Task=`7f93e2fd-2717-45ab-8e1a-e6e794606a21`::moving from state preparing -> state finished >Thread-1248::DEBUG::2014-10-15 07:16:01,336::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1248::DEBUG::2014-10-15 07:16:01,337::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1248::DEBUG::2014-10-15 07:16:01,337::task::993::Storage.TaskManager.Task::(_decref) Task=`7f93e2fd-2717-45ab-8e1a-e6e794606a21`::ref 0 aborting False >Thread-1248::ERROR::2014-10-15 07:16:01,337::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1248::DEBUG::2014-10-15 07:16:01,338::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:01,350::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:01,351::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1249::DEBUG::2014-10-15 07:16:01,352::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1249::DEBUG::2014-10-15 07:16:01,352::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:04,347::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:04,348::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:04,348::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:04,475::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:04,475::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1250::DEBUG::2014-10-15 07:16:04,476::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:04,478::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:04,478::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1251::DEBUG::2014-10-15 07:16:04,479::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1251::DEBUG::2014-10-15 07:16:04,479::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:07,507::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:07,508::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1252::DEBUG::2014-10-15 07:16:07,509::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:07,510::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:07,511::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1253::DEBUG::2014-10-15 07:16:07,512::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1253::DEBUG::2014-10-15 07:16:07,512::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:16:08,189::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:08,189::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:08,189::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:08,189::__init__::232::IOProcess::(_processLogs) (1067) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:16:08,190::__init__::232::IOProcess::(_processLogs) (1067) Queuing response >Thread-25::DEBUG::2014-10-15 07:16:08,190::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:16:08,197::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000423495 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:16:08,198::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:08,198::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:08,198::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:08,198::__init__::232::IOProcess::(_processLogs) (1068) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:16:08,199::__init__::232::IOProcess::(_processLogs) (1068) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:08,199::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:08,199::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:08,199::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:08,199::__init__::232::IOProcess::(_processLogs) (1069) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:08,200::__init__::232::IOProcess::(_processLogs) (1069) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:08,200::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:08,200::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:08,200::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:08,200::__init__::232::IOProcess::(_processLogs) (1070) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:08,200::__init__::232::IOProcess::(_processLogs) (1070) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:08,201::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:08,201::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:08,201::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:08,201::__init__::232::IOProcess::(_processLogs) (1071) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:08,201::__init__::232::IOProcess::(_processLogs) (1071) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:09,352::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:09,352::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:09,353::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:10,551::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:10,552::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1254::DEBUG::2014-10-15 07:16:10,553::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:10,554::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:10,555::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1255::DEBUG::2014-10-15 07:16:10,555::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1255::DEBUG::2014-10-15 07:16:10,556::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:13,577::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:13,577::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1256::DEBUG::2014-10-15 07:16:13,579::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:13,580::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:13,580::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1257::DEBUG::2014-10-15 07:16:13,581::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1257::DEBUG::2014-10-15 07:16:13,582::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:14,358::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:14,359::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:14,359::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:16,600::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:16,600::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1258::DEBUG::2014-10-15 07:16:16,601::task::595::Storage.TaskManager.Task::(_updateState) Task=`3c73eee6-6058-4e88-a3b0-5889ae9aa288`::moving from state init -> state preparing >Thread-1258::INFO::2014-10-15 07:16:16,602::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1258::INFO::2014-10-15 07:16:16,602::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423495', 'lastCheck': '8.4', 'valid': True}} >Thread-1258::DEBUG::2014-10-15 07:16:16,602::task::1191::Storage.TaskManager.Task::(prepare) Task=`3c73eee6-6058-4e88-a3b0-5889ae9aa288`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423495', 'lastCheck': '8.4', 'valid': True}} >Thread-1258::DEBUG::2014-10-15 07:16:16,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`3c73eee6-6058-4e88-a3b0-5889ae9aa288`::moving from state preparing -> state finished >Thread-1258::DEBUG::2014-10-15 07:16:16,602::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1258::DEBUG::2014-10-15 07:16:16,602::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1258::DEBUG::2014-10-15 07:16:16,602::task::993::Storage.TaskManager.Task::(_decref) Task=`3c73eee6-6058-4e88-a3b0-5889ae9aa288`::ref 0 aborting False >Thread-1258::ERROR::2014-10-15 07:16:16,603::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1258::DEBUG::2014-10-15 07:16:16,604::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:16,614::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:16,614::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1259::DEBUG::2014-10-15 07:16:16,615::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1259::DEBUG::2014-10-15 07:16:16,616::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:16:18,202::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:18,202::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:18,202::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:18,202::__init__::232::IOProcess::(_processLogs) (1072) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:16:18,203::__init__::232::IOProcess::(_processLogs) (1072) Queuing response >Thread-25::DEBUG::2014-10-15 07:16:18,203::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:16:18,209::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000401817 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:16:18,210::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:18,210::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:18,210::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:18,210::__init__::232::IOProcess::(_processLogs) (1073) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:16:18,210::__init__::232::IOProcess::(_processLogs) (1073) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:18,211::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:18,211::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:18,211::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:18,211::__init__::232::IOProcess::(_processLogs) (1074) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:18,211::__init__::232::IOProcess::(_processLogs) (1074) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:18,212::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:18,212::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:18,212::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:18,212::__init__::232::IOProcess::(_processLogs) (1075) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:18,212::__init__::232::IOProcess::(_processLogs) (1075) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:18,212::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:18,212::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:18,213::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:18,213::__init__::232::IOProcess::(_processLogs) (1076) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:18,213::__init__::232::IOProcess::(_processLogs) (1076) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:19,365::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:19,366::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:19,366::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:19,738::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:19,738::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1260::DEBUG::2014-10-15 07:16:19,739::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:19,740::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:19,741::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1261::DEBUG::2014-10-15 07:16:19,742::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1261::DEBUG::2014-10-15 07:16:19,742::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:22,768::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:22,769::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1262::DEBUG::2014-10-15 07:16:22,770::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:22,771::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:22,772::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1263::DEBUG::2014-10-15 07:16:22,773::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1263::DEBUG::2014-10-15 07:16:22,773::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:24,372::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:24,372::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:24,373::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:25,794::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:25,795::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1264::DEBUG::2014-10-15 07:16:25,796::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:25,797::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:25,797::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1265::DEBUG::2014-10-15 07:16:25,798::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1265::DEBUG::2014-10-15 07:16:25,798::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-1065::DEBUG::2014-10-15 07:16:26,335::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb2067cc-bc70-4ab9-ad1a-40b284eddd86`::moving from state init -> state preparing >Thread-1065::INFO::2014-10-15 07:16:26,335::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', volUUID='486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:16:26,335::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:26,336::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:26,336::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:26,336::__init__::232::IOProcess::(_processLogs) (1077) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:16:26,336::__init__::232::IOProcess::(_processLogs) (1077) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:26,337::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:26,337::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:26,337::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:26,337::__init__::232::IOProcess::(_processLogs) (1078) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:16:26,337::__init__::232::IOProcess::(_processLogs) (1078) Queuing response >Thread-1065::INFO::2014-10-15 07:16:26,338::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:16:26,338::task::1191::Storage.TaskManager.Task::(prepare) Task=`fb2067cc-bc70-4ab9-ad1a-40b284eddd86`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:16:26,338::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb2067cc-bc70-4ab9-ad1a-40b284eddd86`::moving from state preparing -> state finished >Thread-1065::DEBUG::2014-10-15 07:16:26,338::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1065::DEBUG::2014-10-15 07:16:26,338::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1065::DEBUG::2014-10-15 07:16:26,338::task::993::Storage.TaskManager.Task::(_decref) Task=`fb2067cc-bc70-4ab9-ad1a-40b284eddd86`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 07:16:26,727::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >Thread-16::DEBUG::2014-10-15 07:16:28,214::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:28,214::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:28,214::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:28,214::__init__::232::IOProcess::(_processLogs) (1079) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:16:28,214::__init__::232::IOProcess::(_processLogs) (1079) Queuing response >Thread-25::DEBUG::2014-10-15 07:16:28,215::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:16:28,221::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000353176 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:16:28,222::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:28,222::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:28,222::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:28,222::__init__::232::IOProcess::(_processLogs) (1080) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:16:28,222::__init__::232::IOProcess::(_processLogs) (1080) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:28,223::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:28,223::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:28,223::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:28,223::__init__::232::IOProcess::(_processLogs) (1081) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:28,223::__init__::232::IOProcess::(_processLogs) (1081) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:28,223::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:28,223::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:28,224::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:28,224::__init__::232::IOProcess::(_processLogs) (1082) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:28,224::__init__::232::IOProcess::(_processLogs) (1082) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:28,224::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:28,224::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:28,224::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:28,224::__init__::232::IOProcess::(_processLogs) (1083) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:28,225::__init__::232::IOProcess::(_processLogs) (1083) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:28,822::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:28,823::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1266::DEBUG::2014-10-15 07:16:28,824::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:28,825::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:28,825::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1267::DEBUG::2014-10-15 07:16:28,826::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1267::DEBUG::2014-10-15 07:16:28,827::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:29,378::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:29,379::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:29,379::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:31,840::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:31,842::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1268::DEBUG::2014-10-15 07:16:31,845::task::595::Storage.TaskManager.Task::(_updateState) Task=`8f559d7c-8a44-4453-befa-480e841d9de4`::moving from state init -> state preparing >Thread-1268::INFO::2014-10-15 07:16:31,845::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1268::INFO::2014-10-15 07:16:31,846::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000353176', 'lastCheck': '3.6', 'valid': True}} >Thread-1268::DEBUG::2014-10-15 07:16:31,847::task::1191::Storage.TaskManager.Task::(prepare) Task=`8f559d7c-8a44-4453-befa-480e841d9de4`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000353176', 'lastCheck': '3.6', 'valid': True}} >Thread-1268::DEBUG::2014-10-15 07:16:31,847::task::595::Storage.TaskManager.Task::(_updateState) Task=`8f559d7c-8a44-4453-befa-480e841d9de4`::moving from state preparing -> state finished >Thread-1268::DEBUG::2014-10-15 07:16:31,848::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1268::DEBUG::2014-10-15 07:16:31,848::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1268::DEBUG::2014-10-15 07:16:31,848::task::993::Storage.TaskManager.Task::(_decref) Task=`8f559d7c-8a44-4453-befa-480e841d9de4`::ref 0 aborting False >Thread-1268::ERROR::2014-10-15 07:16:31,851::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1268::DEBUG::2014-10-15 07:16:31,854::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:31,865::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:31,866::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1269::DEBUG::2014-10-15 07:16:31,869::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1269::DEBUG::2014-10-15 07:16:31,870::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:34,384::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:34,385::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:34,385::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:34,960::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:34,961::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1270::DEBUG::2014-10-15 07:16:34,962::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:34,963::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:34,963::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1271::DEBUG::2014-10-15 07:16:34,964::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1271::DEBUG::2014-10-15 07:16:34,965::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:38,015::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:38,016::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1272::DEBUG::2014-10-15 07:16:38,017::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:38,018::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:38,019::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1273::DEBUG::2014-10-15 07:16:38,020::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1273::DEBUG::2014-10-15 07:16:38,020::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:16:38,226::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:38,227::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:38,227::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:38,228::__init__::232::IOProcess::(_processLogs) (1084) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:16:38,228::__init__::232::IOProcess::(_processLogs) (1084) Queuing response >Thread-25::DEBUG::2014-10-15 07:16:38,230::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:16:38,236::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000356255 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:16:38,238::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:38,238::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:38,239::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:38,239::__init__::232::IOProcess::(_processLogs) (1085) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:16:38,240::__init__::232::IOProcess::(_processLogs) (1085) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:38,242::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:38,242::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:38,243::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:38,243::__init__::232::IOProcess::(_processLogs) (1086) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:38,244::__init__::232::IOProcess::(_processLogs) (1086) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:38,245::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:38,246::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:38,246::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:38,247::__init__::232::IOProcess::(_processLogs) (1087) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:38,248::__init__::232::IOProcess::(_processLogs) (1087) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:38,249::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:38,250::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:38,250::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:38,251::__init__::232::IOProcess::(_processLogs) (1088) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:38,251::__init__::232::IOProcess::(_processLogs) (1088) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:39,391::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:39,392::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:39,392::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:41,072::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:41,073::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1274::DEBUG::2014-10-15 07:16:41,074::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:41,075::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:41,076::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1275::DEBUG::2014-10-15 07:16:41,077::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1275::DEBUG::2014-10-15 07:16:41,077::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:44,133::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:44,134::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1276::DEBUG::2014-10-15 07:16:44,135::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:44,136::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:44,136::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1277::DEBUG::2014-10-15 07:16:44,137::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1277::DEBUG::2014-10-15 07:16:44,137::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:44,398::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:44,398::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:44,399::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:47,151::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:47,152::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1278::DEBUG::2014-10-15 07:16:47,153::task::595::Storage.TaskManager.Task::(_updateState) Task=`f419c9d0-8f12-44d9-9761-77951e2593a5`::moving from state init -> state preparing >Thread-1278::INFO::2014-10-15 07:16:47,153::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1278::INFO::2014-10-15 07:16:47,153::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000356255', 'lastCheck': '8.9', 'valid': True}} >Thread-1278::DEBUG::2014-10-15 07:16:47,154::task::1191::Storage.TaskManager.Task::(prepare) Task=`f419c9d0-8f12-44d9-9761-77951e2593a5`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000356255', 'lastCheck': '8.9', 'valid': True}} >Thread-1278::DEBUG::2014-10-15 07:16:47,154::task::595::Storage.TaskManager.Task::(_updateState) Task=`f419c9d0-8f12-44d9-9761-77951e2593a5`::moving from state preparing -> state finished >Thread-1278::DEBUG::2014-10-15 07:16:47,154::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1278::DEBUG::2014-10-15 07:16:47,154::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1278::DEBUG::2014-10-15 07:16:47,154::task::993::Storage.TaskManager.Task::(_decref) Task=`f419c9d0-8f12-44d9-9761-77951e2593a5`::ref 0 aborting False >Thread-1278::ERROR::2014-10-15 07:16:47,155::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1278::DEBUG::2014-10-15 07:16:47,155::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:47,162::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:47,163::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1279::DEBUG::2014-10-15 07:16:47,164::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1279::DEBUG::2014-10-15 07:16:47,164::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:16:48,253::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:48,253::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:48,254::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:48,254::__init__::232::IOProcess::(_processLogs) (1089) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:16:48,254::__init__::232::IOProcess::(_processLogs) (1089) Queuing response >Thread-25::DEBUG::2014-10-15 07:16:48,254::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:16:48,261::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000441624 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:16:48,262::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:48,262::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:48,262::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:48,262::__init__::232::IOProcess::(_processLogs) (1090) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:16:48,262::__init__::232::IOProcess::(_processLogs) (1090) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:48,263::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:48,263::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:48,263::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:48,263::__init__::232::IOProcess::(_processLogs) (1091) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:48,263::__init__::232::IOProcess::(_processLogs) (1091) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:48,263::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:48,263::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:48,264::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:48,264::__init__::232::IOProcess::(_processLogs) (1092) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:48,264::__init__::232::IOProcess::(_processLogs) (1092) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:48,266::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:48,267::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:48,267::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:48,268::__init__::232::IOProcess::(_processLogs) (1093) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:48,268::__init__::232::IOProcess::(_processLogs) (1093) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:49,403::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:49,404::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:49,404::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:50,264::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:50,265::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1280::DEBUG::2014-10-15 07:16:50,266::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:50,267::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:50,267::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1281::DEBUG::2014-10-15 07:16:50,268::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1281::DEBUG::2014-10-15 07:16:50,268::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:53,304::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:53,304::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1282::DEBUG::2014-10-15 07:16:53,305::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:53,306::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:53,307::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1283::DEBUG::2014-10-15 07:16:53,308::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1283::DEBUG::2014-10-15 07:16:53,308::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:54,405::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:54,406::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:54,406::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:56,156::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:56,157::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1284::DEBUG::2014-10-15 07:16:56,158::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:56,159::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:56,160::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1285::DEBUG::2014-10-15 07:16:56,161::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1285::DEBUG::2014-10-15 07:16:56,161::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:16:57,585::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >Thread-16::DEBUG::2014-10-15 07:16:58,270::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:58,270::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:58,270::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:58,271::__init__::232::IOProcess::(_processLogs) (1094) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:16:58,271::__init__::232::IOProcess::(_processLogs) (1094) Queuing response >Thread-25::DEBUG::2014-10-15 07:16:58,271::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:16:58,277::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000392221 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:16:58,278::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:58,278::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:58,278::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:58,278::__init__::232::IOProcess::(_processLogs) (1095) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:16:58,278::__init__::232::IOProcess::(_processLogs) (1095) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:58,279::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:58,279::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:58,279::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:58,279::__init__::232::IOProcess::(_processLogs) (1096) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:58,279::__init__::232::IOProcess::(_processLogs) (1096) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:58,279::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:58,280::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:58,280::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:58,280::__init__::232::IOProcess::(_processLogs) (1097) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:58,280::__init__::232::IOProcess::(_processLogs) (1097) Queuing response >Thread-16::DEBUG::2014-10-15 07:16:58,280::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:16:58,280::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:16:58,280::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:16:58,281::__init__::232::IOProcess::(_processLogs) (1098) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:16:58,281::__init__::232::IOProcess::(_processLogs) (1098) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:59,179::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:59,180::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1286::DEBUG::2014-10-15 07:16:59,181::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:16:59,182::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:16:59,182::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1287::DEBUG::2014-10-15 07:16:59,183::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1287::DEBUG::2014-10-15 07:16:59,184::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:59,233::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:59,234::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:16:59,234::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:02,206::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:02,207::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1288::DEBUG::2014-10-15 07:17:02,210::task::595::Storage.TaskManager.Task::(_updateState) Task=`f67faa52-b4dc-4303-88c1-3d0aa951189a`::moving from state init -> state preparing >Thread-1288::INFO::2014-10-15 07:17:02,212::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1288::INFO::2014-10-15 07:17:02,212::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000392221', 'lastCheck': '3.9', 'valid': True}} >Thread-1288::DEBUG::2014-10-15 07:17:02,213::task::1191::Storage.TaskManager.Task::(prepare) Task=`f67faa52-b4dc-4303-88c1-3d0aa951189a`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000392221', 'lastCheck': '3.9', 'valid': True}} >Thread-1288::DEBUG::2014-10-15 07:17:02,213::task::595::Storage.TaskManager.Task::(_updateState) Task=`f67faa52-b4dc-4303-88c1-3d0aa951189a`::moving from state preparing -> state finished >Thread-1288::DEBUG::2014-10-15 07:17:02,214::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1288::DEBUG::2014-10-15 07:17:02,214::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1288::DEBUG::2014-10-15 07:17:02,215::task::993::Storage.TaskManager.Task::(_decref) Task=`f67faa52-b4dc-4303-88c1-3d0aa951189a`::ref 0 aborting False >Thread-1288::ERROR::2014-10-15 07:17:02,218::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1288::DEBUG::2014-10-15 07:17:02,220::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:02,232::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:02,233::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1289::DEBUG::2014-10-15 07:17:02,236::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1289::DEBUG::2014-10-15 07:17:02,237::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:04,238::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:04,238::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:04,239::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:05,365::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:05,365::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1290::DEBUG::2014-10-15 07:17:05,366::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:05,367::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:05,368::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1291::DEBUG::2014-10-15 07:17:05,369::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1291::DEBUG::2014-10-15 07:17:05,369::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:17:08,282::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:08,282::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:08,282::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:08,282::__init__::232::IOProcess::(_processLogs) (1099) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:17:08,283::__init__::232::IOProcess::(_processLogs) (1099) Queuing response >Thread-25::DEBUG::2014-10-15 07:17:08,283::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:17:08,295::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00174688 s, 263 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:17:08,295::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:08,296::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:08,296::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:08,296::__init__::232::IOProcess::(_processLogs) (1100) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:17:08,296::__init__::232::IOProcess::(_processLogs) (1100) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:08,297::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:08,297::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:08,297::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:08,297::__init__::232::IOProcess::(_processLogs) (1101) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:08,297::__init__::232::IOProcess::(_processLogs) (1101) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:08,297::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:08,297::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:08,298::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:08,298::__init__::232::IOProcess::(_processLogs) (1102) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:08,298::__init__::232::IOProcess::(_processLogs) (1102) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:08,298::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:08,298::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:08,298::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:08,298::__init__::232::IOProcess::(_processLogs) (1103) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:08,298::__init__::232::IOProcess::(_processLogs) (1103) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:08,399::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:08,399::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1292::DEBUG::2014-10-15 07:17:08,400::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:08,402::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:08,402::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1293::DEBUG::2014-10-15 07:17:08,403::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1293::DEBUG::2014-10-15 07:17:08,403::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:09,245::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:09,245::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:09,246::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:11,420::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:11,421::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1294::DEBUG::2014-10-15 07:17:11,422::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:11,423::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:11,423::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1295::DEBUG::2014-10-15 07:17:11,424::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1295::DEBUG::2014-10-15 07:17:11,425::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:14,251::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:14,252::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:14,252::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:14,449::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:14,449::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1296::DEBUG::2014-10-15 07:17:14,450::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:14,451::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:14,452::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1297::DEBUG::2014-10-15 07:17:14,453::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1297::DEBUG::2014-10-15 07:17:14,453::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:17,468::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:17,469::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1298::DEBUG::2014-10-15 07:17:17,471::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b97175d-cf95-4ab7-ad39-8ea61f10eb57`::moving from state init -> state preparing >Thread-1298::INFO::2014-10-15 07:17:17,471::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1298::INFO::2014-10-15 07:17:17,471::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00174688', 'lastCheck': '9.2', 'valid': True}} >Thread-1298::DEBUG::2014-10-15 07:17:17,471::task::1191::Storage.TaskManager.Task::(prepare) Task=`0b97175d-cf95-4ab7-ad39-8ea61f10eb57`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00174688', 'lastCheck': '9.2', 'valid': True}} >Thread-1298::DEBUG::2014-10-15 07:17:17,471::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b97175d-cf95-4ab7-ad39-8ea61f10eb57`::moving from state preparing -> state finished >Thread-1298::DEBUG::2014-10-15 07:17:17,471::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1298::DEBUG::2014-10-15 07:17:17,471::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1298::DEBUG::2014-10-15 07:17:17,471::task::993::Storage.TaskManager.Task::(_decref) Task=`0b97175d-cf95-4ab7-ad39-8ea61f10eb57`::ref 0 aborting False >Thread-1298::ERROR::2014-10-15 07:17:17,472::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1298::DEBUG::2014-10-15 07:17:17,473::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:17,482::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:17,483::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1299::DEBUG::2014-10-15 07:17:17,484::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1299::DEBUG::2014-10-15 07:17:17,484::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:17:18,299::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:18,300::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:18,300::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:18,300::__init__::232::IOProcess::(_processLogs) (1104) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:17:18,300::__init__::232::IOProcess::(_processLogs) (1104) Queuing response >Thread-25::DEBUG::2014-10-15 07:17:18,300::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:17:18,307::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000365975 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:17:18,307::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:18,307::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:18,308::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:18,308::__init__::232::IOProcess::(_processLogs) (1105) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:17:18,308::__init__::232::IOProcess::(_processLogs) (1105) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:18,308::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:18,308::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:18,309::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:18,309::__init__::232::IOProcess::(_processLogs) (1106) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:18,309::__init__::232::IOProcess::(_processLogs) (1106) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:18,309::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:18,309::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:18,309::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:18,309::__init__::232::IOProcess::(_processLogs) (1107) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:18,309::__init__::232::IOProcess::(_processLogs) (1107) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:18,310::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:18,310::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:18,310::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:18,310::__init__::232::IOProcess::(_processLogs) (1108) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:18,310::__init__::232::IOProcess::(_processLogs) (1108) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:19,258::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:19,259::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:19,259::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:20,601::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:20,602::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1300::DEBUG::2014-10-15 07:17:20,603::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:20,604::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:20,605::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1301::DEBUG::2014-10-15 07:17:20,605::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1301::DEBUG::2014-10-15 07:17:20,606::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:23,629::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:23,630::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1302::DEBUG::2014-10-15 07:17:23,631::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:23,632::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:23,633::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1303::DEBUG::2014-10-15 07:17:23,634::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1303::DEBUG::2014-10-15 07:17:23,634::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:24,265::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:24,265::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:24,266::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1065::DEBUG::2014-10-15 07:17:26,375::task::595::Storage.TaskManager.Task::(_updateState) Task=`c67a8990-1169-4b77-878d-3f9898d83d4e`::moving from state init -> state preparing >Thread-1065::INFO::2014-10-15 07:17:26,375::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', volUUID='486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:17:26,376::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:26,376::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:26,376::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:26,376::__init__::232::IOProcess::(_processLogs) (1109) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:17:26,377::__init__::232::IOProcess::(_processLogs) (1109) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:26,377::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:26,377::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:26,378::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:26,378::__init__::232::IOProcess::(_processLogs) (1110) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:17:26,378::__init__::232::IOProcess::(_processLogs) (1110) Queuing response >Thread-1065::INFO::2014-10-15 07:17:26,378::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:17:26,378::task::1191::Storage.TaskManager.Task::(prepare) Task=`c67a8990-1169-4b77-878d-3f9898d83d4e`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:17:26,378::task::595::Storage.TaskManager.Task::(_updateState) Task=`c67a8990-1169-4b77-878d-3f9898d83d4e`::moving from state preparing -> state finished >Thread-1065::DEBUG::2014-10-15 07:17:26,378::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1065::DEBUG::2014-10-15 07:17:26,378::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1065::DEBUG::2014-10-15 07:17:26,378::task::993::Storage.TaskManager.Task::(_decref) Task=`c67a8990-1169-4b77-878d-3f9898d83d4e`::ref 0 aborting False >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:26,649::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:26,650::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1304::DEBUG::2014-10-15 07:17:26,651::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:26,652::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:26,653::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1305::DEBUG::2014-10-15 07:17:26,653::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1305::DEBUG::2014-10-15 07:17:26,654::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:17:27,618::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >Thread-16::DEBUG::2014-10-15 07:17:28,311::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:28,311::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:28,311::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:28,311::__init__::232::IOProcess::(_processLogs) (1111) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:17:28,312::__init__::232::IOProcess::(_processLogs) (1111) Queuing response >Thread-25::DEBUG::2014-10-15 07:17:28,312::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:17:28,318::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000426067 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:17:28,319::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:28,319::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:28,319::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:28,319::__init__::232::IOProcess::(_processLogs) (1112) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:17:28,319::__init__::232::IOProcess::(_processLogs) (1112) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:28,320::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:28,320::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:28,320::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:28,320::__init__::232::IOProcess::(_processLogs) (1113) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:28,320::__init__::232::IOProcess::(_processLogs) (1113) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:28,321::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:28,321::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:28,321::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:28,321::__init__::232::IOProcess::(_processLogs) (1114) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:28,321::__init__::232::IOProcess::(_processLogs) (1114) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:28,321::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:28,321::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:28,322::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:28,322::__init__::232::IOProcess::(_processLogs) (1115) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:28,322::__init__::232::IOProcess::(_processLogs) (1115) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:29,271::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:29,272::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:29,272::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:29,688::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:29,689::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1306::DEBUG::2014-10-15 07:17:29,690::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:29,691::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:29,691::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1307::DEBUG::2014-10-15 07:17:29,692::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1307::DEBUG::2014-10-15 07:17:29,692::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:32,708::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:32,709::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1308::DEBUG::2014-10-15 07:17:32,710::task::595::Storage.TaskManager.Task::(_updateState) Task=`c661ee4d-c935-4e65-aa14-679bc01db545`::moving from state init -> state preparing >Thread-1308::INFO::2014-10-15 07:17:32,711::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1308::INFO::2014-10-15 07:17:32,711::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426067', 'lastCheck': '4.4', 'valid': True}} >Thread-1308::DEBUG::2014-10-15 07:17:32,711::task::1191::Storage.TaskManager.Task::(prepare) Task=`c661ee4d-c935-4e65-aa14-679bc01db545`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426067', 'lastCheck': '4.4', 'valid': True}} >Thread-1308::DEBUG::2014-10-15 07:17:32,711::task::595::Storage.TaskManager.Task::(_updateState) Task=`c661ee4d-c935-4e65-aa14-679bc01db545`::moving from state preparing -> state finished >Thread-1308::DEBUG::2014-10-15 07:17:32,711::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1308::DEBUG::2014-10-15 07:17:32,711::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1308::DEBUG::2014-10-15 07:17:32,711::task::993::Storage.TaskManager.Task::(_decref) Task=`c661ee4d-c935-4e65-aa14-679bc01db545`::ref 0 aborting False >Thread-1308::ERROR::2014-10-15 07:17:32,712::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1308::DEBUG::2014-10-15 07:17:32,713::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:32,722::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:32,723::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1309::DEBUG::2014-10-15 07:17:32,724::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1309::DEBUG::2014-10-15 07:17:32,724::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:34,278::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:34,279::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:34,279::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:36,008::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:36,008::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1310::DEBUG::2014-10-15 07:17:36,009::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:36,010::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:36,010::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1311::DEBUG::2014-10-15 07:17:36,011::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1311::DEBUG::2014-10-15 07:17:36,012::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:17:38,323::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:38,323::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:38,323::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:38,323::__init__::232::IOProcess::(_processLogs) (1116) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:17:38,323::__init__::232::IOProcess::(_processLogs) (1116) Queuing response >Thread-25::DEBUG::2014-10-15 07:17:38,324::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:17:38,330::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00036477 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:17:38,330::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:38,331::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:38,331::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:38,331::__init__::232::IOProcess::(_processLogs) (1117) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:17:38,331::__init__::232::IOProcess::(_processLogs) (1117) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:38,332::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:38,332::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:38,332::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:38,332::__init__::232::IOProcess::(_processLogs) (1118) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:38,332::__init__::232::IOProcess::(_processLogs) (1118) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:38,332::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:38,332::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:38,333::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:38,333::__init__::232::IOProcess::(_processLogs) (1119) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:38,333::__init__::232::IOProcess::(_processLogs) (1119) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:38,333::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:38,333::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:38,333::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:38,334::__init__::232::IOProcess::(_processLogs) (1120) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:38,334::__init__::232::IOProcess::(_processLogs) (1120) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:39,052::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:39,053::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1312::DEBUG::2014-10-15 07:17:39,053::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:39,054::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:39,055::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1313::DEBUG::2014-10-15 07:17:39,056::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1313::DEBUG::2014-10-15 07:17:39,056::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:39,285::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:39,285::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:39,286::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:42,074::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:42,075::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1314::DEBUG::2014-10-15 07:17:42,076::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:42,077::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:42,077::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1315::DEBUG::2014-10-15 07:17:42,078::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1315::DEBUG::2014-10-15 07:17:42,079::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:44,291::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:44,292::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:44,292::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:45,090::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:45,091::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1316::DEBUG::2014-10-15 07:17:45,092::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:45,093::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:45,094::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1317::DEBUG::2014-10-15 07:17:45,094::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1317::DEBUG::2014-10-15 07:17:45,095::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:48,109::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:48,110::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1318::DEBUG::2014-10-15 07:17:48,111::task::595::Storage.TaskManager.Task::(_updateState) Task=`27d3ff93-9169-418b-b88c-5fbc197e4ed0`::moving from state init -> state preparing >Thread-1318::INFO::2014-10-15 07:17:48,111::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1318::INFO::2014-10-15 07:17:48,112::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00036477', 'lastCheck': '9.8', 'valid': True}} >Thread-1318::DEBUG::2014-10-15 07:17:48,112::task::1191::Storage.TaskManager.Task::(prepare) Task=`27d3ff93-9169-418b-b88c-5fbc197e4ed0`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00036477', 'lastCheck': '9.8', 'valid': True}} >Thread-1318::DEBUG::2014-10-15 07:17:48,112::task::595::Storage.TaskManager.Task::(_updateState) Task=`27d3ff93-9169-418b-b88c-5fbc197e4ed0`::moving from state preparing -> state finished >Thread-1318::DEBUG::2014-10-15 07:17:48,112::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1318::DEBUG::2014-10-15 07:17:48,112::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1318::DEBUG::2014-10-15 07:17:48,112::task::993::Storage.TaskManager.Task::(_decref) Task=`27d3ff93-9169-418b-b88c-5fbc197e4ed0`::ref 0 aborting False >Thread-1318::ERROR::2014-10-15 07:17:48,113::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1318::DEBUG::2014-10-15 07:17:48,114::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:48,126::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:48,126::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1319::DEBUG::2014-10-15 07:17:48,127::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1319::DEBUG::2014-10-15 07:17:48,128::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:17:48,335::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:48,335::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:48,335::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:48,335::__init__::232::IOProcess::(_processLogs) (1121) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:17:48,335::__init__::232::IOProcess::(_processLogs) (1121) Queuing response >Thread-25::DEBUG::2014-10-15 07:17:48,335::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:17:48,342::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000435573 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:17:48,342::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:48,342::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:48,343::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:48,343::__init__::232::IOProcess::(_processLogs) (1122) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:17:48,343::__init__::232::IOProcess::(_processLogs) (1122) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:48,343::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:48,343::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:48,344::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:48,344::__init__::232::IOProcess::(_processLogs) (1123) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:48,344::__init__::232::IOProcess::(_processLogs) (1123) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:48,344::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:48,344::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:48,344::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:48,344::__init__::232::IOProcess::(_processLogs) (1124) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:48,344::__init__::232::IOProcess::(_processLogs) (1124) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:48,345::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:48,345::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:48,345::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:48,345::__init__::232::IOProcess::(_processLogs) (1125) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:48,345::__init__::232::IOProcess::(_processLogs) (1125) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:49,298::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:49,299::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:49,299::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:51,259::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:51,260::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1320::DEBUG::2014-10-15 07:17:51,261::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:51,262::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:51,263::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1321::DEBUG::2014-10-15 07:17:51,263::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1321::DEBUG::2014-10-15 07:17:51,264::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:54,298::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:54,299::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1322::DEBUG::2014-10-15 07:17:54,300::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:54,301::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:54,301::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1323::DEBUG::2014-10-15 07:17:54,302::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1323::DEBUG::2014-10-15 07:17:54,303::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:54,305::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:54,305::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:54,305::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:57,325::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:57,325::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1324::DEBUG::2014-10-15 07:17:57,326::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:17:57,328::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:17:57,328::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1325::DEBUG::2014-10-15 07:17:57,329::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1325::DEBUG::2014-10-15 07:17:57,329::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:17:57,650::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >Thread-16::DEBUG::2014-10-15 07:17:58,346::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:58,346::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:58,346::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:58,346::__init__::232::IOProcess::(_processLogs) (1126) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:17:58,347::__init__::232::IOProcess::(_processLogs) (1126) Queuing response >Thread-25::DEBUG::2014-10-15 07:17:58,347::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:17:58,353::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000388885 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:17:58,354::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:58,354::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:58,354::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:58,354::__init__::232::IOProcess::(_processLogs) (1127) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:17:58,354::__init__::232::IOProcess::(_processLogs) (1127) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:58,355::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:58,355::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:58,355::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:58,355::__init__::232::IOProcess::(_processLogs) (1128) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:58,355::__init__::232::IOProcess::(_processLogs) (1128) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:58,356::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:58,356::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:58,356::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:58,356::__init__::232::IOProcess::(_processLogs) (1129) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:58,356::__init__::232::IOProcess::(_processLogs) (1129) Queuing response >Thread-16::DEBUG::2014-10-15 07:17:58,357::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:17:58,357::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:17:58,357::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:17:58,357::__init__::232::IOProcess::(_processLogs) (1130) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:17:58,357::__init__::232::IOProcess::(_processLogs) (1130) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:59,311::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:59,312::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:17:59,312::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:00,350::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:00,351::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1326::DEBUG::2014-10-15 07:18:00,352::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:00,352::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:00,353::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1327::DEBUG::2014-10-15 07:18:00,354::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1327::DEBUG::2014-10-15 07:18:00,354::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:03,371::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:03,371::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1328::DEBUG::2014-10-15 07:18:03,372::task::595::Storage.TaskManager.Task::(_updateState) Task=`9160ba69-bd2a-4870-a5d2-bc5d72279dcf`::moving from state init -> state preparing >Thread-1328::INFO::2014-10-15 07:18:03,373::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1328::INFO::2014-10-15 07:18:03,373::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000388885', 'lastCheck': '5.0', 'valid': True}} >Thread-1328::DEBUG::2014-10-15 07:18:03,373::task::1191::Storage.TaskManager.Task::(prepare) Task=`9160ba69-bd2a-4870-a5d2-bc5d72279dcf`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000388885', 'lastCheck': '5.0', 'valid': True}} >Thread-1328::DEBUG::2014-10-15 07:18:03,373::task::595::Storage.TaskManager.Task::(_updateState) Task=`9160ba69-bd2a-4870-a5d2-bc5d72279dcf`::moving from state preparing -> state finished >Thread-1328::DEBUG::2014-10-15 07:18:03,373::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1328::DEBUG::2014-10-15 07:18:03,373::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1328::DEBUG::2014-10-15 07:18:03,373::task::993::Storage.TaskManager.Task::(_decref) Task=`9160ba69-bd2a-4870-a5d2-bc5d72279dcf`::ref 0 aborting False >Thread-1328::ERROR::2014-10-15 07:18:03,374::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1328::DEBUG::2014-10-15 07:18:03,375::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:03,382::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:03,383::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1329::DEBUG::2014-10-15 07:18:03,384::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1329::DEBUG::2014-10-15 07:18:03,384::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:04,314::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:04,315::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:04,315::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:06,495::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:06,495::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1330::DEBUG::2014-10-15 07:18:06,496::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:06,498::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:06,498::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1331::DEBUG::2014-10-15 07:18:06,499::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1331::DEBUG::2014-10-15 07:18:06,499::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:18:08,358::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:08,358::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:08,358::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:08,359::__init__::232::IOProcess::(_processLogs) (1131) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:18:08,359::__init__::232::IOProcess::(_processLogs) (1131) Queuing response >Thread-25::DEBUG::2014-10-15 07:18:08,360::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:18:08,367::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000380857 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:18:08,368::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:08,368::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:08,368::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:08,368::__init__::232::IOProcess::(_processLogs) (1132) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:18:08,368::__init__::232::IOProcess::(_processLogs) (1132) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:08,369::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:08,369::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:08,369::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:08,369::__init__::232::IOProcess::(_processLogs) (1133) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:08,369::__init__::232::IOProcess::(_processLogs) (1133) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:08,370::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:08,370::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:08,370::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:08,370::__init__::232::IOProcess::(_processLogs) (1134) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:08,370::__init__::232::IOProcess::(_processLogs) (1134) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:08,370::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:08,371::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:08,371::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:08,371::__init__::232::IOProcess::(_processLogs) (1135) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:08,371::__init__::232::IOProcess::(_processLogs) (1135) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:09,321::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:09,321::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:09,322::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:09,560::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:09,561::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1332::DEBUG::2014-10-15 07:18:09,562::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:09,563::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:09,563::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1333::DEBUG::2014-10-15 07:18:09,564::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1333::DEBUG::2014-10-15 07:18:09,564::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:12,580::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:12,581::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1334::DEBUG::2014-10-15 07:18:12,582::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:12,583::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:12,584::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1335::DEBUG::2014-10-15 07:18:12,584::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1335::DEBUG::2014-10-15 07:18:12,585::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:14,327::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:14,328::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:14,328::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:15,639::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:15,640::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1336::DEBUG::2014-10-15 07:18:15,640::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:15,641::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:15,642::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1337::DEBUG::2014-10-15 07:18:15,643::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1337::DEBUG::2014-10-15 07:18:15,643::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:18:18,372::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:18,372::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:18,372::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:18,372::__init__::232::IOProcess::(_processLogs) (1136) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:18:18,372::__init__::232::IOProcess::(_processLogs) (1136) Queuing response >Thread-25::DEBUG::2014-10-15 07:18:18,373::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:18:18,379::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000359682 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:18:18,379::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:18,380::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:18,380::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:18,380::__init__::232::IOProcess::(_processLogs) (1137) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:18:18,380::__init__::232::IOProcess::(_processLogs) (1137) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:18,380::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:18,381::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:18,381::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:18,381::__init__::232::IOProcess::(_processLogs) (1138) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:18,381::__init__::232::IOProcess::(_processLogs) (1138) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:18,381::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:18,381::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:18,381::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:18,382::__init__::232::IOProcess::(_processLogs) (1139) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:18,382::__init__::232::IOProcess::(_processLogs) (1139) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:18,382::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:18,382::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:18,382::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:18,382::__init__::232::IOProcess::(_processLogs) (1140) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:18,382::__init__::232::IOProcess::(_processLogs) (1140) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:18,659::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:18,660::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1338::DEBUG::2014-10-15 07:18:18,661::task::595::Storage.TaskManager.Task::(_updateState) Task=`08b0d20b-eb21-49da-b91a-1b06e12f4289`::moving from state init -> state preparing >Thread-1338::INFO::2014-10-15 07:18:18,661::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1338::INFO::2014-10-15 07:18:18,661::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000359682', 'lastCheck': '0.3', 'valid': True}} >Thread-1338::DEBUG::2014-10-15 07:18:18,662::task::1191::Storage.TaskManager.Task::(prepare) Task=`08b0d20b-eb21-49da-b91a-1b06e12f4289`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000359682', 'lastCheck': '0.3', 'valid': True}} >Thread-1338::DEBUG::2014-10-15 07:18:18,662::task::595::Storage.TaskManager.Task::(_updateState) Task=`08b0d20b-eb21-49da-b91a-1b06e12f4289`::moving from state preparing -> state finished >Thread-1338::DEBUG::2014-10-15 07:18:18,662::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1338::DEBUG::2014-10-15 07:18:18,662::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1338::DEBUG::2014-10-15 07:18:18,662::task::993::Storage.TaskManager.Task::(_decref) Task=`08b0d20b-eb21-49da-b91a-1b06e12f4289`::ref 0 aborting False >Thread-1338::ERROR::2014-10-15 07:18:18,663::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1338::DEBUG::2014-10-15 07:18:18,664::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:18,672::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:18,673::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1339::DEBUG::2014-10-15 07:18:18,674::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1339::DEBUG::2014-10-15 07:18:18,674::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:19,334::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:19,334::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:19,335::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:21,787::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:21,788::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1340::DEBUG::2014-10-15 07:18:21,789::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:21,790::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:21,790::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1341::DEBUG::2014-10-15 07:18:21,791::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1341::DEBUG::2014-10-15 07:18:21,792::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:24,341::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:24,341::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:24,342::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:24,825::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:24,825::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1342::DEBUG::2014-10-15 07:18:24,826::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:24,827::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:24,828::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1343::DEBUG::2014-10-15 07:18:24,829::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1343::DEBUG::2014-10-15 07:18:24,829::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-1065::DEBUG::2014-10-15 07:18:26,418::task::595::Storage.TaskManager.Task::(_updateState) Task=`e63365c1-6a7b-49bb-a361-adde459e928c`::moving from state init -> state preparing >Thread-1065::INFO::2014-10-15 07:18:26,418::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', volUUID='486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:18:26,419::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:26,419::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:26,419::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:26,419::__init__::232::IOProcess::(_processLogs) (1141) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:18:26,420::__init__::232::IOProcess::(_processLogs) (1141) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:26,420::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:26,420::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:26,420::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:26,420::__init__::232::IOProcess::(_processLogs) (1142) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:18:26,421::__init__::232::IOProcess::(_processLogs) (1142) Queuing response >Thread-1065::INFO::2014-10-15 07:18:26,421::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:18:26,421::task::1191::Storage.TaskManager.Task::(prepare) Task=`e63365c1-6a7b-49bb-a361-adde459e928c`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:18:26,421::task::595::Storage.TaskManager.Task::(_updateState) Task=`e63365c1-6a7b-49bb-a361-adde459e928c`::moving from state preparing -> state finished >Thread-1065::DEBUG::2014-10-15 07:18:26,421::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1065::DEBUG::2014-10-15 07:18:26,421::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1065::DEBUG::2014-10-15 07:18:26,421::task::993::Storage.TaskManager.Task::(_decref) Task=`e63365c1-6a7b-49bb-a361-adde459e928c`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 07:18:27,683::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:27,847::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:27,848::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1344::DEBUG::2014-10-15 07:18:27,849::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:27,850::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:27,850::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1345::DEBUG::2014-10-15 07:18:27,851::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1345::DEBUG::2014-10-15 07:18:27,852::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:18:28,383::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:28,383::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:28,383::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:28,384::__init__::232::IOProcess::(_processLogs) (1143) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:18:28,384::__init__::232::IOProcess::(_processLogs) (1143) Queuing response >Thread-25::DEBUG::2014-10-15 07:18:28,384::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:18:28,390::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000406252 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:18:28,391::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:28,391::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:28,391::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:28,391::__init__::232::IOProcess::(_processLogs) (1144) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:18:28,391::__init__::232::IOProcess::(_processLogs) (1144) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:28,392::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:28,392::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:28,392::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:28,392::__init__::232::IOProcess::(_processLogs) (1145) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:28,392::__init__::232::IOProcess::(_processLogs) (1145) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:28,393::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:28,393::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:28,393::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:28,393::__init__::232::IOProcess::(_processLogs) (1146) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:28,393::__init__::232::IOProcess::(_processLogs) (1146) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:28,394::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:28,394::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:28,394::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:28,394::__init__::232::IOProcess::(_processLogs) (1147) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:28,394::__init__::232::IOProcess::(_processLogs) (1147) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:29,347::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:29,348::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:29,348::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:30,874::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:30,875::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1346::DEBUG::2014-10-15 07:18:30,876::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:30,877::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:30,877::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1347::DEBUG::2014-10-15 07:18:30,878::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1347::DEBUG::2014-10-15 07:18:30,879::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:33,900::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:33,901::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1348::DEBUG::2014-10-15 07:18:33,902::task::595::Storage.TaskManager.Task::(_updateState) Task=`732f188b-4b1e-4e29-9585-d6801ddacfff`::moving from state init -> state preparing >Thread-1348::INFO::2014-10-15 07:18:33,902::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1348::INFO::2014-10-15 07:18:33,903::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000406252', 'lastCheck': '5.5', 'valid': True}} >Thread-1348::DEBUG::2014-10-15 07:18:33,903::task::1191::Storage.TaskManager.Task::(prepare) Task=`732f188b-4b1e-4e29-9585-d6801ddacfff`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000406252', 'lastCheck': '5.5', 'valid': True}} >Thread-1348::DEBUG::2014-10-15 07:18:33,903::task::595::Storage.TaskManager.Task::(_updateState) Task=`732f188b-4b1e-4e29-9585-d6801ddacfff`::moving from state preparing -> state finished >Thread-1348::DEBUG::2014-10-15 07:18:33,903::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1348::DEBUG::2014-10-15 07:18:33,903::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1348::DEBUG::2014-10-15 07:18:33,903::task::993::Storage.TaskManager.Task::(_decref) Task=`732f188b-4b1e-4e29-9585-d6801ddacfff`::ref 0 aborting False >Thread-1348::ERROR::2014-10-15 07:18:33,904::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1348::DEBUG::2014-10-15 07:18:33,905::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:33,918::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:33,918::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1349::DEBUG::2014-10-15 07:18:33,919::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1349::DEBUG::2014-10-15 07:18:33,920::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:34,354::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:34,354::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:34,355::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:37,060::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:37,061::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1350::DEBUG::2014-10-15 07:18:37,062::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:37,063::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:37,063::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1351::DEBUG::2014-10-15 07:18:37,064::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1351::DEBUG::2014-10-15 07:18:37,065::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:18:38,395::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:38,396::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:38,397::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:38,397::__init__::232::IOProcess::(_processLogs) (1148) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:18:38,398::__init__::232::IOProcess::(_processLogs) (1148) Queuing response >Thread-25::DEBUG::2014-10-15 07:18:38,399::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:18:38,406::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000361865 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:18:38,407::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:38,408::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:38,409::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:38,409::__init__::232::IOProcess::(_processLogs) (1149) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:18:38,410::__init__::232::IOProcess::(_processLogs) (1149) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:38,412::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:38,412::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:38,413::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:38,413::__init__::232::IOProcess::(_processLogs) (1150) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:38,414::__init__::232::IOProcess::(_processLogs) (1150) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:38,415::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:38,416::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:38,417::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:38,417::__init__::232::IOProcess::(_processLogs) (1151) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:38,418::__init__::232::IOProcess::(_processLogs) (1151) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:38,419::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:38,420::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:38,420::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:38,421::__init__::232::IOProcess::(_processLogs) (1152) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:38,421::__init__::232::IOProcess::(_processLogs) (1152) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:39,361::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:39,361::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:39,361::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:40,100::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:40,101::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1352::DEBUG::2014-10-15 07:18:40,102::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:40,103::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:40,103::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1353::DEBUG::2014-10-15 07:18:40,104::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1353::DEBUG::2014-10-15 07:18:40,104::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:43,123::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:43,123::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1354::DEBUG::2014-10-15 07:18:43,124::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:43,126::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:43,126::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1355::DEBUG::2014-10-15 07:18:43,127::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1355::DEBUG::2014-10-15 07:18:43,127::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:44,367::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:44,368::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:44,368::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:46,149::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:46,150::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1356::DEBUG::2014-10-15 07:18:46,151::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:46,152::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:46,152::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1357::DEBUG::2014-10-15 07:18:46,153::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1357::DEBUG::2014-10-15 07:18:46,154::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:18:48,423::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:48,424::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:48,424::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:48,424::__init__::232::IOProcess::(_processLogs) (1153) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:18:48,424::__init__::232::IOProcess::(_processLogs) (1153) Queuing response >Thread-25::DEBUG::2014-10-15 07:18:48,424::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:18:48,431::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000456314 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:18:48,431::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:48,431::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:48,432::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:48,432::__init__::232::IOProcess::(_processLogs) (1154) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:18:48,432::__init__::232::IOProcess::(_processLogs) (1154) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:48,432::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:48,433::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:48,433::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:48,433::__init__::232::IOProcess::(_processLogs) (1155) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:48,433::__init__::232::IOProcess::(_processLogs) (1155) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:48,433::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:48,433::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:48,433::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:48,433::__init__::232::IOProcess::(_processLogs) (1156) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:48,434::__init__::232::IOProcess::(_processLogs) (1156) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:48,434::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:48,434::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:48,434::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:48,434::__init__::232::IOProcess::(_processLogs) (1157) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:48,434::__init__::232::IOProcess::(_processLogs) (1157) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:49,170::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:49,170::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1358::DEBUG::2014-10-15 07:18:49,172::task::595::Storage.TaskManager.Task::(_updateState) Task=`b26e8923-a5e2-498a-9f1f-912fd7061edb`::moving from state init -> state preparing >Thread-1358::INFO::2014-10-15 07:18:49,172::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1358::INFO::2014-10-15 07:18:49,172::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000456314', 'lastCheck': '0.7', 'valid': True}} >Thread-1358::DEBUG::2014-10-15 07:18:49,172::task::1191::Storage.TaskManager.Task::(prepare) Task=`b26e8923-a5e2-498a-9f1f-912fd7061edb`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000456314', 'lastCheck': '0.7', 'valid': True}} >Thread-1358::DEBUG::2014-10-15 07:18:49,172::task::595::Storage.TaskManager.Task::(_updateState) Task=`b26e8923-a5e2-498a-9f1f-912fd7061edb`::moving from state preparing -> state finished >Thread-1358::DEBUG::2014-10-15 07:18:49,172::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1358::DEBUG::2014-10-15 07:18:49,172::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1358::DEBUG::2014-10-15 07:18:49,173::task::993::Storage.TaskManager.Task::(_decref) Task=`b26e8923-a5e2-498a-9f1f-912fd7061edb`::ref 0 aborting False >Thread-1358::ERROR::2014-10-15 07:18:49,174::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1358::DEBUG::2014-10-15 07:18:49,174::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:49,183::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:49,184::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1359::DEBUG::2014-10-15 07:18:49,185::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1359::DEBUG::2014-10-15 07:18:49,185::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:49,374::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:49,374::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:49,375::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:52,302::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:52,302::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1360::DEBUG::2014-10-15 07:18:52,303::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:52,304::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:52,305::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1361::DEBUG::2014-10-15 07:18:52,306::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1361::DEBUG::2014-10-15 07:18:52,306::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:54,380::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:54,381::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:54,381::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:55,335::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:55,336::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1362::DEBUG::2014-10-15 07:18:55,337::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:55,338::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:55,339::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1363::DEBUG::2014-10-15 07:18:55,340::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1363::DEBUG::2014-10-15 07:18:55,340::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:18:57,715::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:58,367::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:58,368::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1364::DEBUG::2014-10-15 07:18:58,369::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:18:58,370::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:18:58,370::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1365::DEBUG::2014-10-15 07:18:58,371::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1365::DEBUG::2014-10-15 07:18:58,372::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:18:58,435::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:58,435::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:58,435::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:58,435::__init__::232::IOProcess::(_processLogs) (1158) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:18:58,436::__init__::232::IOProcess::(_processLogs) (1158) Queuing response >Thread-25::DEBUG::2014-10-15 07:18:58,436::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:18:58,443::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000449578 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:18:58,443::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:58,443::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:58,444::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:58,444::__init__::232::IOProcess::(_processLogs) (1159) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:18:58,444::__init__::232::IOProcess::(_processLogs) (1159) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:58,444::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:58,444::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:58,444::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:58,445::__init__::232::IOProcess::(_processLogs) (1160) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:58,445::__init__::232::IOProcess::(_processLogs) (1160) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:58,445::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:58,445::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:58,445::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:58,445::__init__::232::IOProcess::(_processLogs) (1161) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:58,446::__init__::232::IOProcess::(_processLogs) (1161) Queuing response >Thread-16::DEBUG::2014-10-15 07:18:58,446::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:18:58,446::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:18:58,446::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:18:58,446::__init__::232::IOProcess::(_processLogs) (1162) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:18:58,446::__init__::232::IOProcess::(_processLogs) (1162) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:59,382::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:59,383::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:18:59,383::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:01,389::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:01,390::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1366::DEBUG::2014-10-15 07:19:01,391::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:01,392::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:01,393::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1367::DEBUG::2014-10-15 07:19:01,394::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1367::DEBUG::2014-10-15 07:19:01,394::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:04,389::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:04,389::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:04,390::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:04,410::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:04,411::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1368::DEBUG::2014-10-15 07:19:04,412::task::595::Storage.TaskManager.Task::(_updateState) Task=`45b62135-dbe7-4136-bde6-b59026a39834`::moving from state init -> state preparing >Thread-1368::INFO::2014-10-15 07:19:04,412::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1368::INFO::2014-10-15 07:19:04,412::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000449578', 'lastCheck': '6.0', 'valid': True}} >Thread-1368::DEBUG::2014-10-15 07:19:04,412::task::1191::Storage.TaskManager.Task::(prepare) Task=`45b62135-dbe7-4136-bde6-b59026a39834`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000449578', 'lastCheck': '6.0', 'valid': True}} >Thread-1368::DEBUG::2014-10-15 07:19:04,412::task::595::Storage.TaskManager.Task::(_updateState) Task=`45b62135-dbe7-4136-bde6-b59026a39834`::moving from state preparing -> state finished >Thread-1368::DEBUG::2014-10-15 07:19:04,413::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1368::DEBUG::2014-10-15 07:19:04,413::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1368::DEBUG::2014-10-15 07:19:04,413::task::993::Storage.TaskManager.Task::(_decref) Task=`45b62135-dbe7-4136-bde6-b59026a39834`::ref 0 aborting False >Thread-1368::ERROR::2014-10-15 07:19:04,414::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1368::DEBUG::2014-10-15 07:19:04,414::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:04,423::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:04,423::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1369::DEBUG::2014-10-15 07:19:04,424::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1369::DEBUG::2014-10-15 07:19:04,425::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:07,550::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:07,550::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1370::DEBUG::2014-10-15 07:19:07,551::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:07,552::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:07,553::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1371::DEBUG::2014-10-15 07:19:07,554::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1371::DEBUG::2014-10-15 07:19:07,554::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:19:08,447::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:08,448::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:08,448::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:08,448::__init__::232::IOProcess::(_processLogs) (1163) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:19:08,449::__init__::232::IOProcess::(_processLogs) (1163) Queuing response >Thread-25::DEBUG::2014-10-15 07:19:08,449::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:19:08,456::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000372294 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:19:08,456::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:08,457::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:08,457::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:08,457::__init__::232::IOProcess::(_processLogs) (1164) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:19:08,457::__init__::232::IOProcess::(_processLogs) (1164) Queuing response >Thread-16::DEBUG::2014-10-15 07:19:08,457::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:08,457::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:08,458::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:08,458::__init__::232::IOProcess::(_processLogs) (1165) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:19:08,458::__init__::232::IOProcess::(_processLogs) (1165) Queuing response >Thread-16::DEBUG::2014-10-15 07:19:08,458::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:08,458::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:08,458::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:08,458::__init__::232::IOProcess::(_processLogs) (1166) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:19:08,458::__init__::232::IOProcess::(_processLogs) (1166) Queuing response >Thread-16::DEBUG::2014-10-15 07:19:08,459::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:08,459::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:08,459::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:08,459::__init__::232::IOProcess::(_processLogs) (1167) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:19:08,459::__init__::232::IOProcess::(_processLogs) (1167) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:09,395::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:09,396::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:09,396::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:10,628::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:10,629::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1372::DEBUG::2014-10-15 07:19:10,630::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:10,631::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:10,631::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1373::DEBUG::2014-10-15 07:19:10,632::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1373::DEBUG::2014-10-15 07:19:10,633::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:13,653::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:13,654::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1374::DEBUG::2014-10-15 07:19:13,655::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:13,656::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:13,657::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1375::DEBUG::2014-10-15 07:19:13,658::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1375::DEBUG::2014-10-15 07:19:13,658::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:14,402::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:14,402::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:14,403::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:16,676::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:16,677::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1376::DEBUG::2014-10-15 07:19:16,678::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:16,679::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:16,680::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1377::DEBUG::2014-10-15 07:19:16,680::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1377::DEBUG::2014-10-15 07:19:16,681::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:19:18,460::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:18,460::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:18,461::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:18,461::__init__::232::IOProcess::(_processLogs) (1168) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:19:18,461::__init__::232::IOProcess::(_processLogs) (1168) Queuing response >Thread-25::DEBUG::2014-10-15 07:19:18,462::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:19:18,468::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000419114 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:19:18,469::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:18,469::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:18,469::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:18,469::__init__::232::IOProcess::(_processLogs) (1169) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:19:18,469::__init__::232::IOProcess::(_processLogs) (1169) Queuing response >Thread-16::DEBUG::2014-10-15 07:19:18,470::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:18,470::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:18,470::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:18,470::__init__::232::IOProcess::(_processLogs) (1170) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:19:18,470::__init__::232::IOProcess::(_processLogs) (1170) Queuing response >Thread-16::DEBUG::2014-10-15 07:19:18,471::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:18,471::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:18,471::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:18,471::__init__::232::IOProcess::(_processLogs) (1171) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:19:18,471::__init__::232::IOProcess::(_processLogs) (1171) Queuing response >Thread-16::DEBUG::2014-10-15 07:19:18,471::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:18,472::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:18,472::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:18,472::__init__::232::IOProcess::(_processLogs) (1172) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:19:18,472::__init__::232::IOProcess::(_processLogs) (1172) Queuing response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:19,409::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:19,409::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:19,410::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:19,698::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:19,699::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1378::DEBUG::2014-10-15 07:19:19,700::task::595::Storage.TaskManager.Task::(_updateState) Task=`a790c356-798e-44e6-afda-a4b2c1bdeae0`::moving from state init -> state preparing >Thread-1378::INFO::2014-10-15 07:19:19,700::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-1378::INFO::2014-10-15 07:19:19,700::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419114', 'lastCheck': '1.2', 'valid': True}} >Thread-1378::DEBUG::2014-10-15 07:19:19,701::task::1191::Storage.TaskManager.Task::(prepare) Task=`a790c356-798e-44e6-afda-a4b2c1bdeae0`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419114', 'lastCheck': '1.2', 'valid': True}} >Thread-1378::DEBUG::2014-10-15 07:19:19,701::task::595::Storage.TaskManager.Task::(_updateState) Task=`a790c356-798e-44e6-afda-a4b2c1bdeae0`::moving from state preparing -> state finished >Thread-1378::DEBUG::2014-10-15 07:19:19,701::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1378::DEBUG::2014-10-15 07:19:19,701::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1378::DEBUG::2014-10-15 07:19:19,701::task::993::Storage.TaskManager.Task::(_decref) Task=`a790c356-798e-44e6-afda-a4b2c1bdeae0`::ref 0 aborting False >Thread-1378::ERROR::2014-10-15 07:19:19,702::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-1378::DEBUG::2014-10-15 07:19:19,702::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:19,711::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:19,712::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1379::DEBUG::2014-10-15 07:19:19,713::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1379::DEBUG::2014-10-15 07:19:19,713::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:22,819::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:22,823::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1380::DEBUG::2014-10-15 07:19:22,825::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:22,826::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:22,827::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1381::DEBUG::2014-10-15 07:19:22,828::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1381::DEBUG::2014-10-15 07:19:22,829::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:24,415::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:24,416::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:24,416::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:25,862::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:25,863::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1382::DEBUG::2014-10-15 07:19:25,864::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:25,865::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:25,866::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1383::DEBUG::2014-10-15 07:19:25,867::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1383::DEBUG::2014-10-15 07:19:25,867::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-1065::DEBUG::2014-10-15 07:19:26,462::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7c1bd72-2285-42e0-93e5-595315ff7b37`::moving from state init -> state preparing >Thread-1065::INFO::2014-10-15 07:19:26,462::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', volUUID='486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:19:26,463::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:26,463::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:26,463::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:26,463::__init__::232::IOProcess::(_processLogs) (1173) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:19:26,464::__init__::232::IOProcess::(_processLogs) (1173) Queuing response >Thread-16::DEBUG::2014-10-15 07:19:26,465::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:26,465::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:26,465::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:26,465::__init__::232::IOProcess::(_processLogs) (1174) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:19:26,465::__init__::232::IOProcess::(_processLogs) (1174) Queuing response >Thread-1065::INFO::2014-10-15 07:19:26,465::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:19:26,466::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7c1bd72-2285-42e0-93e5-595315ff7b37`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-1065::DEBUG::2014-10-15 07:19:26,466::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7c1bd72-2285-42e0-93e5-595315ff7b37`::moving from state preparing -> state finished >Thread-1065::DEBUG::2014-10-15 07:19:26,466::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-1065::DEBUG::2014-10-15 07:19:26,466::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-1065::DEBUG::2014-10-15 07:19:26,466::task::993::Storage.TaskManager.Task::(_decref) Task=`c7c1bd72-2285-42e0-93e5-595315ff7b37`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 07:19:27,748::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 30. >Thread-25::DEBUG::2014-10-15 07:19:28,472::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 07:19:28,473::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:28,474::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:28,474::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:28,474::__init__::232::IOProcess::(_processLogs) (1175) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:19:28,474::__init__::232::IOProcess::(_processLogs) (1175) Queuing response >Thread-25::DEBUG::2014-10-15 07:19:28,481::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 07:19:28,481::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:28,482::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:28,482::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:28,482::__init__::232::IOProcess::(_processLogs) (1176) Got request for method 'touch' >Thread-16::DEBUG::2014-10-15 07:19:28,527::__init__::232::IOProcess::(_processLogs) (1176) Queuing response >Thread-25::DEBUG::2014-10-15 07:19:28,528::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-16::DEBUG::2014-10-15 07:19:28,528::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:28,528::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:28,528::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:28,528::__init__::232::IOProcess::(_processLogs) (1177) Got request for method 'readfile' >Thread-16::DEBUG::2014-10-15 07:19:28,529::__init__::232::IOProcess::(_processLogs) (1177) Queuing response >Thread-25::DEBUG::2014-10-15 07:19:28,529::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=domain1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-0000000003e1', 'REMOTE_PATH=10.66.128.104:/home/NFS1', 'ROLE=Master', 'SDUUID=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b90f33781d11f6e6b0b3945de3167c0719a458c1'] >Thread-16::DEBUG::2014-10-15 07:19:28,529::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:28,530::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:28,530::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:28,530::__init__::232::IOProcess::(_processLogs) (1178) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:19:28,530::__init__::232::IOProcess::(_processLogs) (1178) Queuing response >Thread-16::DEBUG::2014-10-15 07:19:28,530::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:28,530::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:28,530::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:28,530::__init__::232::IOProcess::(_processLogs) (1179) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:19:28,531::__init__::232::IOProcess::(_processLogs) (1179) Queuing response >Thread-25::DEBUG::2014-10-15 07:19:28,531::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-25::INFO::2014-10-15 07:19:28,531::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_imageNS already registered >Thread-25::INFO::2014-10-15 07:19:28,531::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_volumeNS already registered >Thread-16::DEBUG::2014-10-15 07:19:28,531::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:28,531::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:28,531::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:28,532::__init__::232::IOProcess::(_processLogs) (1180) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:19:28,532::__init__::232::IOProcess::(_processLogs) (1180) Queuing response >Thread-25::DEBUG::2014-10-15 07:19:28,532::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-25::DEBUG::2014-10-15 07:19:28,538::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000374847 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:19:28,538::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:28,538::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:28,539::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:28,539::__init__::232::IOProcess::(_processLogs) (1181) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:19:28,539::__init__::232::IOProcess::(_processLogs) (1181) Queuing response >Thread-16::DEBUG::2014-10-15 07:19:28,539::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:28,539::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:28,540::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:28,540::__init__::232::IOProcess::(_processLogs) (1182) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:19:28,540::__init__::232::IOProcess::(_processLogs) (1182) Queuing response >Thread-16::DEBUG::2014-10-15 07:19:28,540::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:28,540::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:28,540::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:28,540::__init__::232::IOProcess::(_processLogs) (1183) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:19:28,540::__init__::232::IOProcess::(_processLogs) (1183) Queuing response >Thread-16::DEBUG::2014-10-15 07:19:28,541::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:28,541::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:28,541::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:28,541::__init__::232::IOProcess::(_processLogs) (1184) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:19:28,541::__init__::232::IOProcess::(_processLogs) (1184) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:28,887::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:28,887::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1385::DEBUG::2014-10-15 07:19:28,889::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:28,890::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:28,890::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1386::DEBUG::2014-10-15 07:19:28,891::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1386::DEBUG::2014-10-15 07:19:28,891::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:29,420::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:29,421::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:19:29,421::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:31,911::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:31,913::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1387::DEBUG::2014-10-15 07:19:31,915::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:19:31,917::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:19:31,919::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-1388::DEBUG::2014-10-15 07:19:31,920::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-1388::DEBUG::2014-10-15 07:19:31,922::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >MainThread::DEBUG::2014-10-15 07:19:32,135::vdsm::58::vds::(sigtermHandler) Received signal 15 >MainThread::DEBUG::2014-10-15 07:19:32,136::protocoldetector::135::vds.MultiProtocolAcceptor::(stop) Stopping Acceptor >Detector thread::DEBUG::2014-10-15 07:19:32,136::protocoldetector::106::vds.MultiProtocolAcceptor::(_cleanup) Cleaning Acceptor >MainThread::INFO::2014-10-15 07:19:32,136::__init__::563::jsonrpc.JsonRpcServer::(stop) Stopping JsonRPC Server >Thread-16::ERROR::2014-10-15 07:19:32,136::__init__::317::IOProcessClient::(_communicate) IOProcess failure >Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 272, in _communicate >Exception: FD closed >MainThread::INFO::2014-10-15 07:19:32,137::vmchannels::188::vds::(stop) VM channels listener was stopped. >Thread-16::DEBUG::2014-10-15 07:19:32,138::__init__::159::IOProcessClient::(_run) Starting IOProcess... >MainThread::INFO::2014-10-15 07:19:32,138::momIF::91::MOM::(stop) Shutting down MOM >MainThread::DEBUG::2014-10-15 07:19:32,138::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c989832-0b92-4b0a-994a-6325a438d3e1`::moving from state init -> state preparing >MainThread::INFO::2014-10-15 07:19:32,143::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-1390::DEBUG::2014-10-15 07:19:32,143::__init__::232::IOProcess::(_processLogs) Closing unrelated FDs... >Thread-11::DEBUG::2014-10-15 07:19:32,144::storageServer::706::Storage.ConnectionMonitor::(_monitorConnections) Monitoring stopped >Thread-1390::DEBUG::2014-10-15 07:19:32,144::__init__::232::IOProcess::(_processLogs) Opening communication channels... >VM Channels Listener::INFO::2014-10-15 07:19:32,753::vmchannels::183::vds::(run) VM channels listener thread has ended. >Thread-34::INFO::2014-10-15 07:19:33,944::storage_mailbox::509::Storage.MailBox.HsmMailMonitor::(run) HSM_MailboxMonitor - Incoming mail monitoring thread stopped, clearing outgoing mail >Thread-34::INFO::2014-10-15 07:19:33,944::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/usr/bin/dd', 'of=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=16'] >Thread-34::DEBUG::2014-10-15 07:19:33,945::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /usr/bin/dd of=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=16 (cwd None) >Thread-34::DEBUG::2014-10-15 07:19:34,048::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.0985966 s, 41.5 kB/s\n'; <rc> = 0 >MainThread::INFO::2014-10-15 07:19:34,795::domainMonitor::137::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::INFO::2014-10-15 07:19:34,795::domainMonitor::114::Storage.DomainMonitor::(stopMonitoring) Stop monitoring 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-25::DEBUG::2014-10-15 07:19:34,795::domainMonitor::192::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-25::INFO::2014-10-15 07:19:34,795::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 (id: 2) >Thread-25::DEBUG::2014-10-15 07:19:37,796::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 released successfully (id: 2) >MainThread::DEBUG::2014-10-15 07:19:37,796::taskManager::90::Storage.TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2014-10-15 07:19:37,796::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2014-10-15 07:19:37,796::task::1191::Storage.TaskManager.Task::(prepare) Task=`8c989832-0b92-4b0a-994a-6325a438d3e1`::finished: None >MainThread::DEBUG::2014-10-15 07:19:37,797::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c989832-0b92-4b0a-994a-6325a438d3e1`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-15 07:19:37,797::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-15 07:19:37,797::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-15 07:19:37,797::task::993::Storage.TaskManager.Task::(_decref) Task=`8c989832-0b92-4b0a-994a-6325a438d3e1`::ref 0 aborting False >MainThread::INFO::2014-10-15 07:19:37,797::vdsm::137::vds::(run) VDSM main thread ended. Waiting for 15 other threads... >MainThread::INFO::2014-10-15 07:19:37,797::vdsm::141::vds::(run) <Thread(libvirtEventLoop, started daemon 139678774568704)> >MainThread::INFO::2014-10-15 07:19:37,797::vdsm::141::vds::(run) <WorkerThread(Thread-6, started daemon 139678724212480)> >MainThread::INFO::2014-10-15 07:19:37,797::vdsm::141::vds::(run) <WorkerThread(Thread-5, started daemon 139678732605184)> >MainThread::INFO::2014-10-15 07:19:37,797::vdsm::141::vds::(run) <WorkerThread(Thread-3, started daemon 139678749390592)> >MainThread::INFO::2014-10-15 07:19:37,797::vdsm::141::vds::(run) <Thread(Thread-1390, started daemon 139677475378944)> >MainThread::INFO::2014-10-15 07:19:37,797::vdsm::141::vds::(run) <WorkerThread(Thread-1, started daemon 139678766176000)> >MainThread::INFO::2014-10-15 07:19:37,797::vdsm::141::vds::(run) <WorkerThread(Thread-10, started daemon 139678216865536)> >MainThread::INFO::2014-10-15 07:19:37,798::vdsm::141::vds::(run) <WorkerThread(Thread-9, started daemon 139678225258240)> >MainThread::INFO::2014-10-15 07:19:37,798::vdsm::141::vds::(run) <WorkerThread(Thread-8, started daemon 139678233650944)> >MainThread::INFO::2014-10-15 07:19:37,798::vdsm::141::vds::(run) <WorkerThread(Thread-7, started daemon 139678242043648)> >MainThread::INFO::2014-10-15 07:19:37,798::vdsm::141::vds::(run) <_MainThread(MainThread, started 139679011252032)> >MainThread::INFO::2014-10-15 07:19:37,798::momIF::91::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-10-15 07:19:37,798::vdsm::141::vds::(run) <MomThread(MOM, started 139677792093952)> >MainThread::INFO::2014-10-15 07:19:37,798::vdsm::141::vds::(run) <WorkerThread(Thread-4, started daemon 139678740997888)> >MainThread::DEBUG::2014-10-15 07:19:37,798::sampling::412::vm.Vm::(stop) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stop statistics collection >Thread-1065::DEBUG::2014-10-15 07:19:37,798::sampling::443::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stats thread finished >MainThread::INFO::2014-10-15 07:19:37,798::vdsm::141::vds::(run) <VmStatsThread(Thread-1065, started daemon 139676463642368)> >MainThread::INFO::2014-10-15 07:19:37,799::vdsm::141::vds::(run) <PolicyEngine(PolicyEngine, started daemon 139677092800256)> >MainThread::INFO::2014-10-15 07:19:37,799::vdsm::141::vds::(run) <WorkerThread(Thread-2, started daemon 139678757783296)> >MainThread::INFO::2014-10-15 07:19:46,167::vdsm::131::vds::(run) (PID: 18833) I am the actual vdsm 4.16.6-1.el7 hp-z220-04.qe.lab.eng.nay.redhat.com (3.10.0-123.8.1.el7.x86_64) >MainThread::DEBUG::2014-10-15 07:19:46,168::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2014-10-15 07:19:46,168::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 10, waitTimeout: 3, maxTasks: 500 >MainThread::DEBUG::2014-10-15 07:19:46,169::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt >MainThread::WARNING::2014-10-15 07:19:46,170::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-10-15 07:19:46,184::supervdsm::77::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2014-10-15 07:19:46,187::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-10-15 07:19:46,188::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) /usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type (cwd None) >MainThread::DEBUG::2014-10-15 07:19:46,210::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-10-15 07:19:46,211::hsm::442::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-10-15 07:19:46,220::hsm::474::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-10-15 07:19:46,220::hsm::475::Storage.HSM::(__cleanStorageRepository) Mount list: ['/rhev/data-center', '/rhev/data-center/mnt/10.66.128.104:_home_NFS1'] >MainThread::DEBUG::2014-10-15 07:19:46,220::hsm::477::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-10-15 07:19:46,221::hsm::520::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >storageRefresh::DEBUG::2014-10-15 07:19:46,222::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2014-10-15 07:19:46,223::dispatcher::46::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-10-15 07:19:46,223::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-15 07:19:46,259::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >storageRefresh::DEBUG::2014-10-15 07:19:46,260::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-15 07:19:46,260::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >storageRefresh::DEBUG::2014-10-15 07:19:46,265::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >MainThread::DEBUG::2014-10-15 07:19:46,265::task::595::Storage.TaskManager.Task::(_updateState) Task=`594b0d7f-4265-4603-82a4-d47b41b43133`::moving from state init -> state preparing >MainThread::INFO::2014-10-15 07:19:46,265::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF object at 0x177a750>>) >MainThread::INFO::2014-10-15 07:19:46,269::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::DEBUG::2014-10-15 07:19:46,269::task::1191::Storage.TaskManager.Task::(prepare) Task=`594b0d7f-4265-4603-82a4-d47b41b43133`::finished: None >MainThread::DEBUG::2014-10-15 07:19:46,269::task::595::Storage.TaskManager.Task::(_updateState) Task=`594b0d7f-4265-4603-82a4-d47b41b43133`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-15 07:19:46,269::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-15 07:19:46,269::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-15 07:19:46,269::task::993::Storage.TaskManager.Task::(_decref) Task=`594b0d7f-4265-4603-82a4-d47b41b43133`::ref 0 aborting False >MainThread::INFO::2014-10-15 07:19:46,270::momIF::49::MOM::(__init__) Starting up MOM >MainThread::INFO::2014-10-15 07:19:46,271::vmchannels::192::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2014-10-15 07:19:46,275::vmchannels::175::vds::(run) Starting VM channels listener thread. >storageRefresh::DEBUG::2014-10-15 07:19:46,277::misc::751::Storage.SamplingMethod::(__call__) Returning last result >clientIFinit::DEBUG::2014-10-15 07:19:46,285::libvirtconnection::150::root::(get) trying to connect libvirt >MainThread::DEBUG::2014-10-15 07:19:46,285::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <rpc.BindingXMLRPC.XmlDetector instance at 0x1da2b48> >storageRefresh::DEBUG::2014-10-15 07:19:46,290::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >storageRefresh::DEBUG::2014-10-15 07:19:46,320::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-15 07:19:46,321::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-15 07:19:46,329::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-15 07:19:46,334::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-15 07:19:46,347::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-15 07:19:46,349::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-15 07:19:46,350::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-15 07:19:46,350::misc::751::Storage.SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2014-10-15 07:19:46,350::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <yajsonrpc.stompReactor.StompDetector instance at 0x1ef65a8> >storageRefresh::DEBUG::2014-10-15 07:19:46,351::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >JsonRpcServer::DEBUG::2014-10-15 07:19:46,352::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Detector thread::DEBUG::2014-10-15 07:19:46,353::protocoldetector::70::vds.MultiProtocolAcceptor::(serve_forever) Acceptor running >Detector thread::DEBUG::2014-10-15 07:19:46,354::protocoldetector::72::vds.MultiProtocolAcceptor::(serve_forever) Using required_size=7 >storageRefresh::DEBUG::2014-10-15 07:19:46,399::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size (cwd None) >clientIFinit::DEBUG::2014-10-15 07:19:46,401::clientIF::494::vds::(_recoverVm) Trying to recover c255610f-42c2-426d-9270-e4497f4a0f37 >clientIFinit::INFO::2014-10-15 07:19:46,406::clientIF::390::vds::(createVm) vmContainerLock acquired by vm c255610f-42c2-426d-9270-e4497f4a0f37 >clientIFinit::DEBUG::2014-10-15 07:19:46,412::libvirtconnection::150::root::(get) trying to connect libvirt >clientIFinit::DEBUG::2014-10-15 07:19:46,438::clientIF::403::vds::(createVm) Total desktops after creation of c255610f-42c2-426d-9270-e4497f4a0f37 is 1 >Thread-13::DEBUG::2014-10-15 07:19:46,438::vm::2259::vm.Vm::(_startUnderlyingVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Start >Thread-13::DEBUG::2014-10-15 07:19:46,439::vm::2263::vm.Vm::(_startUnderlyingVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::_ongoingCreations acquired >Thread-13::INFO::2014-10-15 07:19:46,440::vm::3299::vm.Vm::(_run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::VM wrapper has started >Thread-13::WARNING::2014-10-15 07:19:46,441::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': 'unix', 'alias': 'channel0', 'deviceType': 'channel', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '1'}}' found >Thread-13::WARNING::2014-10-15 07:19:46,442::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': 'unix', 'alias': 'channel1', 'deviceType': 'channel', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '2'}}' found >Thread-13::WARNING::2014-10-15 07:19:46,442::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': 'spicevmc', 'alias': 'channel2', 'deviceType': 'channel', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '3'}}' found >Thread-13::DEBUG::2014-10-15 07:19:46,444::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4fc878c-1df7-4e3c-ad6d-ee248eedf578`::moving from state init -> state preparing >Thread-13::INFO::2014-10-15 07:19:46,444::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', volUUID='486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-13::ERROR::2014-10-15 07:19:46,445::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-13::ERROR::2014-10-15 07:19:46,445::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-13::DEBUG::2014-10-15 07:19:46,446::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >Thread-13::DEBUG::2014-10-15 07:19:46,447::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 (cwd None) >storageRefresh::DEBUG::2014-10-15 07:19:46,683::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-15 07:19:46,683::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >storageRefresh::DEBUG::2014-10-15 07:19:46,683::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-13::DEBUG::2014-10-15 07:19:46,713::lvm::288::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume group "55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4" not found\n Skipping volume group 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4\n'; <rc> = 5 >Thread-13::WARNING::2014-10-15 07:19:46,714::lvm::370::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4" not found', ' Skipping volume group 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4'] >Thread-15::DEBUG::2014-10-15 07:19:46,715::__init__::159::IOProcessClient::(_run) Starting IOProcess... >storageRefresh::DEBUG::2014-10-15 07:19:46,721::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:19:46,721::__init__::232::IOProcess::(_processLogs) Closing unrelated FDs... >storageRefresh::DEBUG::2014-10-15 07:19:46,722::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-16::DEBUG::2014-10-15 07:19:46,722::__init__::232::IOProcess::(_processLogs) Opening communication channels... >storageRefresh::DEBUG::2014-10-15 07:19:46,723::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags (cwd None) >Thread-16::DEBUG::2014-10-15 07:19:46,723::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:46,728::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:46,729::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:46,729::__init__::232::IOProcess::(_processLogs) (1) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:19:46,730::__init__::232::IOProcess::(_processLogs) (1) Queuing response >Thread-13::DEBUG::2014-10-15 07:19:46,737::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 07:19:46,737::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:46,737::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:46,738::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:46,738::__init__::232::IOProcess::(_processLogs) (2) Got request for method 'touch' >storageRefresh::DEBUG::2014-10-15 07:19:46,754::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-15 07:19:46,755::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Data >storageRefresh::DEBUG::2014-10-15 07:19:46,755::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Logging >storageRefresh::DEBUG::2014-10-15 07:19:46,755::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Config >storageRefresh::DEBUG::2014-10-15 07:19:46,755::lvm::653::Storage.LVM::(bootstrap) Skipping open lv: vg=HostVG lv=Swap >storageRefresh::DEBUG::2014-10-15 07:19:46,755::hsm::387::Storage.HSM::(storageRefresh) HSM is ready >Thread-16::DEBUG::2014-10-15 07:19:46,762::__init__::232::IOProcess::(_processLogs) (2) Queuing response >Thread-13::DEBUG::2014-10-15 07:19:46,763::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-16::DEBUG::2014-10-15 07:19:46,763::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:46,763::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:46,764::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:46,764::__init__::232::IOProcess::(_processLogs) (3) Got request for method 'readfile' >Thread-16::DEBUG::2014-10-15 07:19:46,764::__init__::232::IOProcess::(_processLogs) (3) Queuing response >Thread-13::DEBUG::2014-10-15 07:19:46,764::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=domain1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-0000000003e1', 'REMOTE_PATH=10.66.128.104:/home/NFS1', 'ROLE=Master', 'SDUUID=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b90f33781d11f6e6b0b3945de3167c0719a458c1'] >Thread-16::DEBUG::2014-10-15 07:19:46,765::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:46,765::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:46,765::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:46,765::__init__::232::IOProcess::(_processLogs) (4) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:19:46,766::__init__::232::IOProcess::(_processLogs) (4) Queuing response >Thread-16::DEBUG::2014-10-15 07:19:46,766::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:46,766::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:46,766::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:46,766::__init__::232::IOProcess::(_processLogs) (5) Got request for method 'glob' >Thread-13::DEBUG::2014-10-15 07:19:46,766::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::DEBUG::2014-10-15 07:19:46,766::__init__::232::IOProcess::(_processLogs) (5) Queuing response >Thread-13::DEBUG::2014-10-15 07:19:46,767::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_imageNS' >Thread-13::DEBUG::2014-10-15 07:19:46,767::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_volumeNS' >Thread-16::DEBUG::2014-10-15 07:19:46,767::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:46,768::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:19:46,768::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:19:46,768::__init__::232::IOProcess::(_processLogs) (6) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:19:46,768::__init__::232::IOProcess::(_processLogs) (6) Queuing response >Thread-16::DEBUG::2014-10-15 07:19:46,768::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:19:46,769::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-13::INFO::2014-10-15 07:19:46,769::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-16::DEBUG::2014-10-15 07:19:46,769::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-13::DEBUG::2014-10-15 07:19:46,770::task::1191::Storage.TaskManager.Task::(prepare) Task=`f4fc878c-1df7-4e3c-ad6d-ee248eedf578`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-16::DEBUG::2014-10-15 07:19:46,770::__init__::232::IOProcess::(_processLogs) (7) Got request for method 'stat' >Thread-13::DEBUG::2014-10-15 07:19:46,770::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4fc878c-1df7-4e3c-ad6d-ee248eedf578`::moving from state preparing -> state finished >Thread-16::DEBUG::2014-10-15 07:19:46,770::__init__::232::IOProcess::(_processLogs) (7) Queuing response >Thread-13::DEBUG::2014-10-15 07:19:46,770::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-13::DEBUG::2014-10-15 07:19:46,771::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-13::DEBUG::2014-10-15 07:19:46,771::task::993::Storage.TaskManager.Task::(_decref) Task=`f4fc878c-1df7-4e3c-ad6d-ee248eedf578`::ref 0 aborting False >Thread-13::DEBUG::2014-10-15 07:19:46,773::vm::1067::vm.Vm::(__init__) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Ignoring param (target, 1048576) in BalloonDevice >Thread-13::DEBUG::2014-10-15 07:19:46,788::vm::5301::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Looking for drive with attributes {'name': u'hdc', 'bootOrder': '', 'boot': [], 'readonly': True, 'address': {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, 'path': '', 'type': u'cdrom'} >Thread-13::DEBUG::2014-10-15 07:19:46,788::vm::5322::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'hdc', u'hdc'), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, True), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}), 'path': ('', ''), 'type': (u'cdrom', u'cdrom')} >Thread-13::DEBUG::2014-10-15 07:19:46,788::vm::5341::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'hdc', None), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, None), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, None), 'path': ('', None), 'type': (u'cdrom', None)} >Thread-13::DEBUG::2014-10-15 07:19:46,789::vm::5301::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Looking for drive with attributes {'name': u'vda', 'bootOrder': u'2', 'boot': [<DOM Element: boot at 0x7f3dbc0bf3b0>], 'readonly': False, 'address': {u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, 'path': u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'type': u'disk'} >Thread-13::DEBUG::2014-10-15 07:19:46,789::vm::5322::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'vda', u'vda'), 'bootOrder': (u'2', u'2'), 'boot': ([<DOM Element: boot at 0x7f3dbc0bf3b0>], None), 'readonly': (False, False), 'address': ({u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, {u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}), 'path': (u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb'), 'type': (u'disk', u'disk')} >Thread-13::DEBUG::2014-10-15 07:19:46,789::vm::5341::vm.Vm::(_getUnderlyingDriveInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Matched {'name': (u'vda', None), 'bootOrder': (u'2', None), 'boot': ([<DOM Element: boot at 0x7f3dbc0bf3b0>], None), 'readonly': (False, None), 'address': ({u'slot': u'0x05', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, None), 'path': (u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', None), 'type': (u'disk', None)} >Thread-13::DEBUG::2014-10-15 07:19:46,824::sampling::405::vm.Vm::(start) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Start statistics collection >Thread-17::DEBUG::2014-10-15 07:19:46,824::sampling::434::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stats thread started >Thread-13::DEBUG::2014-10-15 07:19:46,826::vmchannels::199::vds::(register) Add fileno 33 to listener's channels. >Thread-13::WARNING::2014-10-15 07:19:46,827::vm::3971::vm.Vm::(_readPauseCode) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::_readPauseCode unsupported by libvirt vm >Thread-13::DEBUG::2014-10-15 07:19:46,831::vm::2289::vm.Vm::(_startUnderlyingVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::_ongoingCreations released >Thread-17::DEBUG::2014-10-15 07:19:46,833::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 74 edom: 10 level: 2 message: argument unsupported: QEMU driver does not support <metadata> element >Thread-17::ERROR::2014-10-15 07:19:46,834::vm::373::vm.Vm::(_sampleCpuTune) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::libvirt does not support metadata >VM Channels Listener::DEBUG::2014-10-15 07:19:47,285::vmchannels::107::vds::(_do_add_channels) fileno 33 was added to unconnected channels. >VM Channels Listener::DEBUG::2014-10-15 07:19:47,285::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 33. >VM Channels Listener::DEBUG::2014-10-15 07:19:47,285::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:19:47,286::guestagent::203::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Connected to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:19:47,286::vmchannels::149::vds::(_handle_unconnected) Connecting to fileno 33 succeeded. >clientIFinit::DEBUG::2014-10-15 07:19:47,441::task::595::Storage.TaskManager.Task::(_updateState) Task=`624f5821-6e08-48f0-a8ce-4577ec010b99`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:19:47,441::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:19:47,441::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:19:47,441::task::1191::Storage.TaskManager.Task::(prepare) Task=`624f5821-6e08-48f0-a8ce-4577ec010b99`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:19:47,441::task::595::Storage.TaskManager.Task::(_updateState) Task=`624f5821-6e08-48f0-a8ce-4577ec010b99`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:19:47,442::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:19:47,442::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:19:47,442::task::993::Storage.TaskManager.Task::(_decref) Task=`624f5821-6e08-48f0-a8ce-4577ec010b99`::ref 0 aborting False >clientIFinit::DEBUG::2014-10-15 07:19:52,447::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e76ff98-3603-4577-a6af-b5c914f674d1`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:19:52,447::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:19:52,448::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:19:52,448::task::1191::Storage.TaskManager.Task::(prepare) Task=`0e76ff98-3603-4577-a6af-b5c914f674d1`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:19:52,448::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e76ff98-3603-4577-a6af-b5c914f674d1`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:19:52,448::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:19:52,448::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:19:52,448::task::993::Storage.TaskManager.Task::(_decref) Task=`0e76ff98-3603-4577-a6af-b5c914f674d1`::ref 0 aborting False >clientIFinit::DEBUG::2014-10-15 07:19:57,454::task::595::Storage.TaskManager.Task::(_updateState) Task=`90ac4da8-0d69-4690-b327-8413ece8f5d3`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:19:57,454::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:19:57,454::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:19:57,454::task::1191::Storage.TaskManager.Task::(prepare) Task=`90ac4da8-0d69-4690-b327-8413ece8f5d3`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:19:57,455::task::595::Storage.TaskManager.Task::(_updateState) Task=`90ac4da8-0d69-4690-b327-8413ece8f5d3`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:19:57,455::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:19:57,455::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:19:57,455::task::993::Storage.TaskManager.Task::(_decref) Task=`90ac4da8-0d69-4690-b327-8413ece8f5d3`::ref 0 aborting False >clientIFinit::DEBUG::2014-10-15 07:20:02,460::task::595::Storage.TaskManager.Task::(_updateState) Task=`50a86ce1-f176-4f70-b884-2f73f3076c5c`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:20:02,461::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:20:02,461::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:02,461::task::1191::Storage.TaskManager.Task::(prepare) Task=`50a86ce1-f176-4f70-b884-2f73f3076c5c`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:02,461::task::595::Storage.TaskManager.Task::(_updateState) Task=`50a86ce1-f176-4f70-b884-2f73f3076c5c`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:20:02,461::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:20:02,461::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:20:02,461::task::993::Storage.TaskManager.Task::(_decref) Task=`50a86ce1-f176-4f70-b884-2f73f3076c5c`::ref 0 aborting False >clientIFinit::DEBUG::2014-10-15 07:20:07,467::task::595::Storage.TaskManager.Task::(_updateState) Task=`dceb2ad5-9177-444c-a26e-8aed03f1c8f7`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:20:07,467::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:20:07,467::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:07,467::task::1191::Storage.TaskManager.Task::(prepare) Task=`dceb2ad5-9177-444c-a26e-8aed03f1c8f7`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:07,467::task::595::Storage.TaskManager.Task::(_updateState) Task=`dceb2ad5-9177-444c-a26e-8aed03f1c8f7`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:20:07,468::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:20:07,468::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:20:07,468::task::993::Storage.TaskManager.Task::(_decref) Task=`dceb2ad5-9177-444c-a26e-8aed03f1c8f7`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:11,378::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:11,379::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:11,379::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:20:12,473::task::595::Storage.TaskManager.Task::(_updateState) Task=`f8888a87-854f-4a72-acdf-f1c864965c75`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:20:12,473::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:20:12,474::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:12,474::task::1191::Storage.TaskManager.Task::(prepare) Task=`f8888a87-854f-4a72-acdf-f1c864965c75`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:12,474::task::595::Storage.TaskManager.Task::(_updateState) Task=`f8888a87-854f-4a72-acdf-f1c864965c75`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:20:12,474::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:20:12,474::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:20:12,474::task::993::Storage.TaskManager.Task::(_decref) Task=`f8888a87-854f-4a72-acdf-f1c864965c75`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:16,385::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:16,385::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:16,385::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:20:17,316::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 33. >clientIFinit::DEBUG::2014-10-15 07:20:17,480::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ea98d14-29e1-468f-8bc2-89f8a5740d73`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:20:17,480::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:20:17,480::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:17,480::task::1191::Storage.TaskManager.Task::(prepare) Task=`2ea98d14-29e1-468f-8bc2-89f8a5740d73`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:17,480::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ea98d14-29e1-468f-8bc2-89f8a5740d73`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:20:17,481::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:20:17,481::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:20:17,481::task::993::Storage.TaskManager.Task::(_decref) Task=`2ea98d14-29e1-468f-8bc2-89f8a5740d73`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:21,390::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:21,391::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:21,391::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:20:22,486::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a931221-ce8a-4049-b3ce-32216c3a5dae`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:20:22,486::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:20:22,487::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:22,487::task::1191::Storage.TaskManager.Task::(prepare) Task=`5a931221-ce8a-4049-b3ce-32216c3a5dae`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:22,487::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a931221-ce8a-4049-b3ce-32216c3a5dae`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:20:22,487::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:20:22,487::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:20:22,487::task::993::Storage.TaskManager.Task::(_decref) Task=`5a931221-ce8a-4049-b3ce-32216c3a5dae`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:26,397::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:26,397::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:26,398::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:20:27,492::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb381dd5-55d4-4565-8b4d-f7d73ce07eae`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:20:27,492::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:20:27,493::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:27,493::task::1191::Storage.TaskManager.Task::(prepare) Task=`bb381dd5-55d4-4565-8b4d-f7d73ce07eae`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:27,493::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb381dd5-55d4-4565-8b4d-f7d73ce07eae`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:20:27,493::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:20:27,493::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:20:27,493::task::993::Storage.TaskManager.Task::(_decref) Task=`bb381dd5-55d4-4565-8b4d-f7d73ce07eae`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:31,404::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:31,404::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:31,404::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:20:32,497::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ebe1782-2185-461d-896f-6a1699aca8fa`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:20:32,497::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:20:32,498::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:32,498::task::1191::Storage.TaskManager.Task::(prepare) Task=`2ebe1782-2185-461d-896f-6a1699aca8fa`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:32,498::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ebe1782-2185-461d-896f-6a1699aca8fa`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:20:32,498::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:20:32,498::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:20:32,498::task::993::Storage.TaskManager.Task::(_decref) Task=`2ebe1782-2185-461d-896f-6a1699aca8fa`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:36,410::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:36,411::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:36,411::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:20:37,504::task::595::Storage.TaskManager.Task::(_updateState) Task=`92ea6b50-0504-4488-b436-e2b41e384cb9`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:20:37,504::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:20:37,505::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:37,505::task::1191::Storage.TaskManager.Task::(prepare) Task=`92ea6b50-0504-4488-b436-e2b41e384cb9`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:37,505::task::595::Storage.TaskManager.Task::(_updateState) Task=`92ea6b50-0504-4488-b436-e2b41e384cb9`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:20:37,505::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:20:37,505::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:20:37,505::task::993::Storage.TaskManager.Task::(_decref) Task=`92ea6b50-0504-4488-b436-e2b41e384cb9`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:41,412::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:41,413::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:41,413::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:20:42,507::task::595::Storage.TaskManager.Task::(_updateState) Task=`08306f19-feec-4c24-b401-5ba8a78bc317`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:20:42,507::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:20:42,507::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:42,507::task::1191::Storage.TaskManager.Task::(prepare) Task=`08306f19-feec-4c24-b401-5ba8a78bc317`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:42,507::task::595::Storage.TaskManager.Task::(_updateState) Task=`08306f19-feec-4c24-b401-5ba8a78bc317`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:20:42,507::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:20:42,508::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:20:42,508::task::993::Storage.TaskManager.Task::(_decref) Task=`08306f19-feec-4c24-b401-5ba8a78bc317`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:46,419::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:46,419::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:46,420::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:20:47,349::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 33. >clientIFinit::DEBUG::2014-10-15 07:20:47,513::task::595::Storage.TaskManager.Task::(_updateState) Task=`a22093b4-be53-421a-82b0-a2a7ba512c11`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:20:47,513::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:20:47,514::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:47,514::task::1191::Storage.TaskManager.Task::(prepare) Task=`a22093b4-be53-421a-82b0-a2a7ba512c11`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:47,514::task::595::Storage.TaskManager.Task::(_updateState) Task=`a22093b4-be53-421a-82b0-a2a7ba512c11`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:20:47,514::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:20:47,514::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:20:47,514::task::993::Storage.TaskManager.Task::(_decref) Task=`a22093b4-be53-421a-82b0-a2a7ba512c11`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:51,425::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:51,426::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:51,426::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:20:52,520::task::595::Storage.TaskManager.Task::(_updateState) Task=`3e6a1a52-61b7-45b8-ac6a-40e2a570305a`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:20:52,520::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:20:52,520::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:52,521::task::1191::Storage.TaskManager.Task::(prepare) Task=`3e6a1a52-61b7-45b8-ac6a-40e2a570305a`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:52,521::task::595::Storage.TaskManager.Task::(_updateState) Task=`3e6a1a52-61b7-45b8-ac6a-40e2a570305a`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:20:52,521::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:20:52,521::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:20:52,521::task::993::Storage.TaskManager.Task::(_decref) Task=`3e6a1a52-61b7-45b8-ac6a-40e2a570305a`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:56,432::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:56,432::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:20:56,433::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:20:57,526::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd109597-fe3e-44e6-9d7e-4d0d856f9733`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:20:57,527::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:20:57,527::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:57,527::task::1191::Storage.TaskManager.Task::(prepare) Task=`bd109597-fe3e-44e6-9d7e-4d0d856f9733`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:20:57,527::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd109597-fe3e-44e6-9d7e-4d0d856f9733`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:20:57,527::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:20:57,528::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:20:57,528::task::993::Storage.TaskManager.Task::(_decref) Task=`bd109597-fe3e-44e6-9d7e-4d0d856f9733`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:01,437::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:01,438::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:01,438::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:21:02,533::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f23b98a-94e5-49ba-966b-862f47b17d58`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:21:02,533::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:21:02,534::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:02,534::task::1191::Storage.TaskManager.Task::(prepare) Task=`9f23b98a-94e5-49ba-966b-862f47b17d58`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:02,534::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f23b98a-94e5-49ba-966b-862f47b17d58`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:21:02,534::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:21:02,534::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:21:02,534::task::993::Storage.TaskManager.Task::(_decref) Task=`9f23b98a-94e5-49ba-966b-862f47b17d58`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:06,444::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:06,444::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:06,445::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:21:07,540::task::595::Storage.TaskManager.Task::(_updateState) Task=`f23104e9-f846-43e2-9cfd-35a10bb4e088`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:21:07,540::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:21:07,540::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:07,541::task::1191::Storage.TaskManager.Task::(prepare) Task=`f23104e9-f846-43e2-9cfd-35a10bb4e088`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:07,541::task::595::Storage.TaskManager.Task::(_updateState) Task=`f23104e9-f846-43e2-9cfd-35a10bb4e088`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:21:07,541::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:21:07,541::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:21:07,541::task::993::Storage.TaskManager.Task::(_decref) Task=`f23104e9-f846-43e2-9cfd-35a10bb4e088`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:11,451::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:11,451::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:11,451::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:21:12,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`89b7d3b2-b501-423c-87be-cf0e5c4b2acc`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:21:12,542::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:21:12,543::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:12,543::task::1191::Storage.TaskManager.Task::(prepare) Task=`89b7d3b2-b501-423c-87be-cf0e5c4b2acc`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:12,543::task::595::Storage.TaskManager.Task::(_updateState) Task=`89b7d3b2-b501-423c-87be-cf0e5c4b2acc`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:21:12,543::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:21:12,543::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:21:12,543::task::993::Storage.TaskManager.Task::(_decref) Task=`89b7d3b2-b501-423c-87be-cf0e5c4b2acc`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:16,456::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:16,456::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:16,457::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:21:17,382::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 33. >clientIFinit::DEBUG::2014-10-15 07:21:17,549::task::595::Storage.TaskManager.Task::(_updateState) Task=`6282344f-0ea3-4fb6-92be-89f3dc63738c`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:21:17,549::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:21:17,549::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:17,549::task::1191::Storage.TaskManager.Task::(prepare) Task=`6282344f-0ea3-4fb6-92be-89f3dc63738c`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:17,550::task::595::Storage.TaskManager.Task::(_updateState) Task=`6282344f-0ea3-4fb6-92be-89f3dc63738c`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:21:17,550::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:21:17,550::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:21:17,550::task::993::Storage.TaskManager.Task::(_decref) Task=`6282344f-0ea3-4fb6-92be-89f3dc63738c`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:21,461::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:21,462::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:21,462::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:21:22,551::task::595::Storage.TaskManager.Task::(_updateState) Task=`1357e5b8-5f87-4585-a1e3-ccde52a2ca8b`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:21:22,552::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:21:22,552::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:22,552::task::1191::Storage.TaskManager.Task::(prepare) Task=`1357e5b8-5f87-4585-a1e3-ccde52a2ca8b`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:22,552::task::595::Storage.TaskManager.Task::(_updateState) Task=`1357e5b8-5f87-4585-a1e3-ccde52a2ca8b`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:21:22,552::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:21:22,552::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:21:22,553::task::993::Storage.TaskManager.Task::(_decref) Task=`1357e5b8-5f87-4585-a1e3-ccde52a2ca8b`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:26,468::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:26,468::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:26,469::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:21:27,558::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f30b8bc-2486-4a1d-a830-fed7c761e59d`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:21:27,558::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:21:27,559::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:27,559::task::1191::Storage.TaskManager.Task::(prepare) Task=`9f30b8bc-2486-4a1d-a830-fed7c761e59d`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:27,559::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f30b8bc-2486-4a1d-a830-fed7c761e59d`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:21:27,559::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:21:27,559::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:21:27,559::task::993::Storage.TaskManager.Task::(_decref) Task=`9f30b8bc-2486-4a1d-a830-fed7c761e59d`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:31,472::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:31,472::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:31,473::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:21:32,564::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea6ae7f0-a19a-4df9-82af-102cee360621`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:21:32,564::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:21:32,565::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:32,565::task::1191::Storage.TaskManager.Task::(prepare) Task=`ea6ae7f0-a19a-4df9-82af-102cee360621`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:32,565::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea6ae7f0-a19a-4df9-82af-102cee360621`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:21:32,565::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:21:32,565::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:21:32,565::task::993::Storage.TaskManager.Task::(_decref) Task=`ea6ae7f0-a19a-4df9-82af-102cee360621`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:36,474::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:36,475::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:36,475::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:21:37,571::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b7af969-f6d7-4fc4-9054-9dc75a1d2591`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:21:37,571::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:21:37,572::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:37,572::task::1191::Storage.TaskManager.Task::(prepare) Task=`8b7af969-f6d7-4fc4-9054-9dc75a1d2591`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:37,572::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b7af969-f6d7-4fc4-9054-9dc75a1d2591`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:21:37,572::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:21:37,572::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:21:37,572::task::993::Storage.TaskManager.Task::(_decref) Task=`8b7af969-f6d7-4fc4-9054-9dc75a1d2591`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:41,481::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:41,481::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:41,482::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:21:42,578::task::595::Storage.TaskManager.Task::(_updateState) Task=`674b743d-0116-41b8-ae1a-ac50edc82b91`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:21:42,578::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:21:42,578::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:42,578::task::1191::Storage.TaskManager.Task::(prepare) Task=`674b743d-0116-41b8-ae1a-ac50edc82b91`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:42,578::task::595::Storage.TaskManager.Task::(_updateState) Task=`674b743d-0116-41b8-ae1a-ac50edc82b91`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:21:42,578::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:21:42,579::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:21:42,579::task::993::Storage.TaskManager.Task::(_decref) Task=`674b743d-0116-41b8-ae1a-ac50edc82b91`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:46,487::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:46,488::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:46,488::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:21:47,413::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 33. >clientIFinit::DEBUG::2014-10-15 07:21:47,584::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2ae2385-363c-4e71-861f-65adc8209e7c`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:21:47,584::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:21:47,585::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:47,585::task::1191::Storage.TaskManager.Task::(prepare) Task=`e2ae2385-363c-4e71-861f-65adc8209e7c`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:47,585::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2ae2385-363c-4e71-861f-65adc8209e7c`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:21:47,585::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:21:47,585::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:21:47,585::task::993::Storage.TaskManager.Task::(_decref) Task=`e2ae2385-363c-4e71-861f-65adc8209e7c`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:51,494::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:51,494::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:51,495::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:21:52,591::task::595::Storage.TaskManager.Task::(_updateState) Task=`edee1ab3-b0f4-40a0-afb7-35ee64314721`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:21:52,591::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:21:52,591::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:52,591::task::1191::Storage.TaskManager.Task::(prepare) Task=`edee1ab3-b0f4-40a0-afb7-35ee64314721`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:52,592::task::595::Storage.TaskManager.Task::(_updateState) Task=`edee1ab3-b0f4-40a0-afb7-35ee64314721`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:21:52,592::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:21:52,592::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:21:52,592::task::993::Storage.TaskManager.Task::(_decref) Task=`edee1ab3-b0f4-40a0-afb7-35ee64314721`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:56,500::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:56,501::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:21:56,501::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:21:57,597::task::595::Storage.TaskManager.Task::(_updateState) Task=`375e03a7-930b-451e-9a09-863aebc9af3f`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:21:57,598::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:21:57,598::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:57,598::task::1191::Storage.TaskManager.Task::(prepare) Task=`375e03a7-930b-451e-9a09-863aebc9af3f`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:21:57,598::task::595::Storage.TaskManager.Task::(_updateState) Task=`375e03a7-930b-451e-9a09-863aebc9af3f`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:21:57,598::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:21:57,598::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:21:57,599::task::993::Storage.TaskManager.Task::(_decref) Task=`375e03a7-930b-451e-9a09-863aebc9af3f`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:01,507::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:01,507::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:01,508::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:22:02,604::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1949cbd-7880-412b-8b82-57a7f8e94aaf`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:22:02,604::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:22:02,605::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:02,605::task::1191::Storage.TaskManager.Task::(prepare) Task=`d1949cbd-7880-412b-8b82-57a7f8e94aaf`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:02,605::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1949cbd-7880-412b-8b82-57a7f8e94aaf`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:22:02,605::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:22:02,605::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:22:02,605::task::993::Storage.TaskManager.Task::(_decref) Task=`d1949cbd-7880-412b-8b82-57a7f8e94aaf`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:06,513::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:06,514::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:06,514::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:22:07,607::task::595::Storage.TaskManager.Task::(_updateState) Task=`09d8c049-7878-4d4f-9187-2f78491114d6`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:22:07,607::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:22:07,608::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:07,608::task::1191::Storage.TaskManager.Task::(prepare) Task=`09d8c049-7878-4d4f-9187-2f78491114d6`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:07,608::task::595::Storage.TaskManager.Task::(_updateState) Task=`09d8c049-7878-4d4f-9187-2f78491114d6`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:22:07,608::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:22:07,608::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:22:07,609::task::993::Storage.TaskManager.Task::(_decref) Task=`09d8c049-7878-4d4f-9187-2f78491114d6`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:11,517::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:11,518::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:11,518::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:22:12,614::task::595::Storage.TaskManager.Task::(_updateState) Task=`51d68958-6d1a-4def-a236-16005d87414d`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:22:12,614::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:22:12,615::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:12,615::task::1191::Storage.TaskManager.Task::(prepare) Task=`51d68958-6d1a-4def-a236-16005d87414d`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:12,615::task::595::Storage.TaskManager.Task::(_updateState) Task=`51d68958-6d1a-4def-a236-16005d87414d`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:22:12,615::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:22:12,615::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:22:12,615::task::993::Storage.TaskManager.Task::(_decref) Task=`51d68958-6d1a-4def-a236-16005d87414d`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:16,524::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:16,524::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:16,525::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-10-15 07:22:17,444::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 33. >clientIFinit::DEBUG::2014-10-15 07:22:17,621::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2c2751b-445a-4921-bfc4-c758f8226674`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:22:17,621::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:22:17,622::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:17,622::task::1191::Storage.TaskManager.Task::(prepare) Task=`e2c2751b-445a-4921-bfc4-c758f8226674`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:17,622::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2c2751b-445a-4921-bfc4-c758f8226674`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:22:17,622::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:22:17,622::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:22:17,622::task::993::Storage.TaskManager.Task::(_decref) Task=`e2c2751b-445a-4921-bfc4-c758f8226674`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:21,530::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:21,531::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:21,531::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:22:22,626::task::595::Storage.TaskManager.Task::(_updateState) Task=`c6b26bdd-d692-46ec-b21e-0e68b85f9754`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:22:22,626::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:22:22,627::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:22,627::task::1191::Storage.TaskManager.Task::(prepare) Task=`c6b26bdd-d692-46ec-b21e-0e68b85f9754`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:22,627::task::595::Storage.TaskManager.Task::(_updateState) Task=`c6b26bdd-d692-46ec-b21e-0e68b85f9754`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:22:22,627::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:22:22,627::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:22:22,627::task::993::Storage.TaskManager.Task::(_decref) Task=`c6b26bdd-d692-46ec-b21e-0e68b85f9754`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:26,536::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:26,537::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:26,537::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:22:27,633::task::595::Storage.TaskManager.Task::(_updateState) Task=`98e3fa16-7cbb-4cbe-b783-33f05cf6f214`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:22:27,633::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:22:27,634::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:27,634::task::1191::Storage.TaskManager.Task::(prepare) Task=`98e3fa16-7cbb-4cbe-b783-33f05cf6f214`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:27,634::task::595::Storage.TaskManager.Task::(_updateState) Task=`98e3fa16-7cbb-4cbe-b783-33f05cf6f214`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:22:27,634::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:22:27,634::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:22:27,634::task::993::Storage.TaskManager.Task::(_decref) Task=`98e3fa16-7cbb-4cbe-b783-33f05cf6f214`::ref 0 aborting False >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:31,543::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:31,543::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >GuestMonitor-rhevh-guest-rhel7.0::DEBUG::2014-10-15 07:22:31,544::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >clientIFinit::DEBUG::2014-10-15 07:22:32,639::task::595::Storage.TaskManager.Task::(_updateState) Task=`58022050-f5fa-4ac9-a144-e0c2b276c739`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:22:32,639::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:22:32,640::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:32,640::task::1191::Storage.TaskManager.Task::(prepare) Task=`58022050-f5fa-4ac9-a144-e0c2b276c739`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:32,640::task::595::Storage.TaskManager.Task::(_updateState) Task=`58022050-f5fa-4ac9-a144-e0c2b276c739`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:22:32,640::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:22:32,640::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:22:32,640::task::993::Storage.TaskManager.Task::(_decref) Task=`58022050-f5fa-4ac9-a144-e0c2b276c739`::ref 0 aborting False >Detector thread::DEBUG::2014-10-15 07:22:35,324::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.66.78.3:37590 >Detector thread::DEBUG::2014-10-15 07:22:35,325::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.66.78.3:37590 >Detector thread::DEBUG::2014-10-15 07:22:35,325::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol stomp from 10.66.78.3:37590 >Detector thread::DEBUG::2014-10-15 07:22:35,325::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='CONNECT'> >Detector thread::INFO::2014-10-15 07:22:35,325::stompReactor::66::Broker.StompAdapter::(_cmd_connect) Processing CONNECT request >Detector thread::DEBUG::2014-10-15 07:22:35,326::stompReactor::338::protocoldetector.StompDetector::(handleSocket) Stomp detected from ('10.66.78.3', 37590) >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:35,326::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SUBSCRIBE'> >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:35,326::stompReactor::89::Broker.StompAdapter::(_cmd_subscribe) Subscribe command ignored >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:35,366::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-18::DEBUG::2014-10-15 07:22:35,367::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'VM.migrate' in bridge with {u'params': {u'tunneled': u'false', u'dstqemu': u'10.66.100.111', u'src': u'10.66.100.109', u'dst': u'10.66.100.111:54321', u'vmId': u'c255610f-42c2-426d-9270-e4497f4a0f37', u'abortOnError': u'true', u'method': u'online'}, u'vmID': u'c255610f-42c2-426d-9270-e4497f4a0f37'} >JsonRpcServer::DEBUG::2014-10-15 07:22:35,367::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-18::DEBUG::2014-10-15 07:22:35,368::API::510::vds::(migrate) {u'tunneled': u'false', u'dstqemu': u'10.66.100.111', u'src': u'10.66.100.109', u'dst': u'10.66.100.111:54321', u'vmId': u'c255610f-42c2-426d-9270-e4497f4a0f37', u'abortOnError': u'true', u'method': u'online'} >Thread-18::DEBUG::2014-10-15 07:22:35,368::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'VM.migrate' in bridge with True >Thread-18::DEBUG::2014-10-15 07:22:35,369::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-19::DEBUG::2014-10-15 07:22:35,371::migration::102::vm.Vm::(_setupVdsConnection) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Destination server is: 10.66.100.111:54321 >Thread-19::DEBUG::2014-10-15 07:22:35,371::migration::104::vm.Vm::(_setupVdsConnection) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Initiating connection with destination >Thread-19::DEBUG::2014-10-15 07:22:35,380::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-19::DEBUG::2014-10-15 07:22:35,381::migration::154::vm.Vm::(_prepareGuest) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Migration started >Thread-19::DEBUG::2014-10-15 07:22:35,389::migration::237::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::migration semaphore acquired after 0 seconds >Thread-19::DEBUG::2014-10-15 07:22:35,527::migration::297::vm.Vm::(_startUnderlyingMigration) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::starting migration to qemu+tls://10.66.100.111/system with miguri tcp://10.66.100.111 >Thread-20::DEBUG::2014-10-15 07:22:35,527::migration::360::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::migration downtime thread started >Thread-21::DEBUG::2014-10-15 07:22:35,528::migration::409::vm.Vm::(monitor_migration) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::starting migration monitor thread >clientIFinit::DEBUG::2014-10-15 07:22:37,646::task::595::Storage.TaskManager.Task::(_updateState) Task=`963589a6-4bbc-4a42-bc86-c84797afcddb`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:22:37,646::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:22:37,647::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:37,647::task::1191::Storage.TaskManager.Task::(prepare) Task=`963589a6-4bbc-4a42-bc86-c84797afcddb`::finished: {'poollist': []} >clientIFinit::DEBUG::2014-10-15 07:22:37,647::task::595::Storage.TaskManager.Task::(_updateState) Task=`963589a6-4bbc-4a42-bc86-c84797afcddb`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:22:37,647::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:22:37,647::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:22:37,647::task::993::Storage.TaskManager.Task::(_decref) Task=`963589a6-4bbc-4a42-bc86-c84797afcddb`::ref 0 aborting False >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:38,274::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:38,275::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-22::DEBUG::2014-10-15 07:22:38,275::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getCapabilities' in bridge with {} >Thread-22::DEBUG::2014-10-15 07:22:38,297::utils::738::root::(execCmd) /sbin/ip route show to 0.0.0.0/0 table all (cwd None) >Thread-22::DEBUG::2014-10-15 07:22:38,304::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-22::DEBUG::2014-10-15 07:22:38,328::utils::738::root::(execCmd) numactl --hardware (cwd None) >Thread-22::DEBUG::2014-10-15 07:22:38,334::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-22::DEBUG::2014-10-15 07:22:38,334::utils::738::root::(execCmd) sysctl -n -e kernel.numa_balancing (cwd None) >Thread-22::DEBUG::2014-10-15 07:22:38,339::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-22::DEBUG::2014-10-15 07:22:38,343::caps::269::root::(getLiveMergeSupport) libvirt is missing 'VIR_DOMAIN_BLOCK_COMMIT_RELATIVE': live merge disabled >Thread-22::DEBUG::2014-10-15 07:22:38,344::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getCapabilities' in bridge with {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:67bf3ab0cc3a'}], 'FC': []}, 'packages2': {'kernel': {'release': '123.8.1.el7.x86_64', 'buildtime': 1407764269.0, 'version': '3.10.0'}, 'spice-server': {'release': '5.el7', 'buildtime': 1391017388L, 'version': '0.12.4'}, 'vdsm': {'release': '1.el7', 'buildtime': 1412523943L, 'version': '4.16.6'}, 'qemu-kvm': {'release': '60.el7_0.10', 'buildtime': 1412003348L, 'version': '1.5.3'}, 'libvirt': {'release': '29.el7_0.3', 'buildtime': 1411483944L, 'version': '1.1.1'}, 'qemu-img': {'release': '60.el7_0.10', 'buildtime': 1412003348L, 'version': '1.5.3'}, 'mom': {'release': '1.el7_0', 'buildtime': 1405326698L, 'version': '0.4.1'}}, 'numaNodeDistance': {'0': [10]}, 'cpuModel': 'Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz', 'liveMerge': 'false', 'hooks': {'before_vm_migrate_destination': {'50_vhostmd': {'md5': 'f3ee6dbf6fbd01333bd3e32afec4fbba'}}, 'after_vm_destroy': {'50_vhostmd': {'md5': '92498ed80c0219749829ecce2813fc7c'}}, 'before_vm_dehibernate': {'50_vhostmd': {'md5': 'f3ee6dbf6fbd01333bd3e32afec4fbba'}}, 'before_vm_start': {'50_hostedengine': {'md5': '45dde62155b5412eafbfff5ef265acc2'}, '50_vhostmd': {'md5': 'f3ee6dbf6fbd01333bd3e32afec4fbba'}}}, 'cpuSockets': '1', 'vmTypes': ['kvm'], 'selinux': {'mode': '1'}, 'kdumpStatus': 0, 'supportedProtocols': ['2.2', '2.3'], 'networks': {'rhevm': {'iface': 'rhevm', 'addr': '10.66.100.109', 'cfg': {'IPV6_AUTOCONF': 'no', 'IPV6INIT': 'no', 'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'PEERDNS': 'no', 'DEVICE': 'rhevm', 'PEERNTP': 'yes', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'bridged': True, 'ipv6addrs': ['fe80::a2b3:ccff:fef7:e533/64'], 'gateway': '10.66.103.254', 'bootproto4': 'dhcp', 'netmask': '255.255.252.0', 'stp': 'off', 'ipv4addrs': ['10.66.100.109/22'], 'mtu': '1500', 'ipv6gateway': '::', 'ports': ['eno1', 'vnet0']}}, 'bridges': {'rhevm': {'addr': '10.66.100.109', 'cfg': {'IPV6_AUTOCONF': 'no', 'IPV6INIT': 'no', 'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'PEERDNS': 'no', 'DEVICE': 'rhevm', 'PEERNTP': 'yes', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ipv6addrs': ['fe80::a2b3:ccff:fef7:e533/64'], 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'ipv4addrs': ['10.66.100.109/22'], 'ipv6gateway': '::', 'gateway': '10.66.103.254', 'opts': {'multicast_last_member_count': '2', 'hash_elasticity': '4', 'multicast_query_response_interval': '1000', 'group_fwd_mask': '0x0', 'multicast_snooping': '1', 'multicast_startup_query_interval': '3125', 'hello_timer': '108', 'multicast_querier_interval': '25500', 'max_age': '2000', 'hash_max': '512', 'stp_state': '0', 'topology_change_detected': '0', 'priority': '32768', 'multicast_membership_interval': '26000', 'root_path_cost': '0', 'root_port': '0', 'multicast_querier': '0', 'multicast_startup_query_count': '2', 'nf_call_iptables': '0', 'topology_change': '0', 'hello_time': '200', 'root_id': '8000.a0b3ccf7e533', 'bridge_id': '8000.a0b3ccf7e533', 'topology_change_timer': '0', 'ageing_time': '30000', 'nf_call_ip6tables': '0', 'gc_timer': '10540', 'nf_call_arptables': '0', 'group_addr': '1:80:c2:0:0:0', 'multicast_last_member_interval': '100', 'multicast_query_interval': '12500', 'tcn_timer': '0', 'multicast_router': '1', 'vlan_filtering': '0', 'forward_delay': '0'}, 'ports': ['eno1', 'vnet0']}}, 'uuid': 'A63BD200-CD0E-11E1-9290-A0B3CCF7E533', 'lastClientIface': 'rhevm', 'nics': {'eno1': {'addr': '', 'cfg': {'BRIDGE': 'rhevm', 'IPV6INIT': 'no', 'NM_CONTROLLED': 'no', 'PEERDNS': 'no', 'IPV6_AUTOCONF': 'no', 'HWADDR': 'a0:b3:cc:f7:e5:33', 'DEVICE': 'eno1', 'PEERNTP': 'yes', 'ONBOOT': 'yes'}, 'ipv6addrs': ['fe80::a2b3:ccff:fef7:e533/64'], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': 'a0:b3:cc:f7:e5:33', 'speed': 1000}}, 'software_revision': '1', 'clusterLevels': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,lahf_lm,ida,arat,epb,xsaveopt,pln,pts,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,fsgsbase,smep,erms,model_Nehalem,model_Conroe,model_coreduo,model_core2duo,model_Penryn,model_Westmere,model_n270,model_SandyBridge', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:67bf3ab0cc3a', 'netConfigDirty': 'False', 'supportedENGINEs': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'autoNumaBalancing': 0, 'reservedMem': '321', 'bondings': {}, 'software_version': '4.16', 'memSize': '7732', 'cpuSpeed': '3414.000', 'numaNodes': {u'0': {'totalMemory': '7732', 'cpus': [0, 1, 2, 3]}}, 'version_name': 'Snow Man', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0'], 'cpuThreads': '4', 'emulatedMachines': [u'pc', u'q35', u'rhel6.0.0', u'rhel6.1.0', u'rhel6.2.0', u'rhel6.3.0', u'rhel6.4.0', u'rhel6.5.0', u'none'], 'rngSources': ['random'], 'operatingSystem': {'release': '20141006.0.el7ev', 'version': '7.0', 'name': 'RHEV Hypervisor'}} >Thread-22::DEBUG::2014-10-15 07:22:38,345::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:38,352::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-23::DEBUG::2014-10-15 07:22:38,352::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getHardwareInfo' in bridge with {} >JsonRpcServer::DEBUG::2014-10-15 07:22:38,352::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-23::DEBUG::2014-10-15 07:22:38,354::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getHardwareInfo' in bridge with {'systemProductName': 'HP Z220 CMT Workstation', 'systemUUID': 'A63BD200-CD0E-11E1-9290-A0B3CCF7E533', 'systemSerialNumber': 'CNG229WDQR', 'systemFamily': '103C_53335X G=D', 'systemManufacturer': 'Hewlett-Packard'} >Thread-23::DEBUG::2014-10-15 07:22:38,354::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:38,382::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:38,382::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-24::DEBUG::2014-10-15 07:22:38,383::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:38,384::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:38,385::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-25::DEBUG::2014-10-15 07:22:38,386::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-25::DEBUG::2014-10-15 07:22:38,386::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:38,419::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:38,419::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-26::DEBUG::2014-10-15 07:22:38,419::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connectStorageServer' in bridge with {u'connectionParams': [{u'id': u'cd8fa491-c53f-4601-8cc6-cc31a63494e7', u'connection': u'10.66.128.104:/home/NFS1', u'iqn': u'', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u''}], u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1', u'domainType': 1} >Thread-26::DEBUG::2014-10-15 07:22:38,420::task::595::Storage.TaskManager.Task::(_updateState) Task=`b3b0d5df-c09e-48a7-888b-a0e4b8bd1b28`::moving from state init -> state preparing >Thread-26::INFO::2014-10-15 07:22:38,420::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID=u'00000002-0002-0002-0002-0000000003e1', conList=[{u'connection': u'10.66.128.104:/home/NFS1', u'iqn': u'', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'cd8fa491-c53f-4601-8cc6-cc31a63494e7', u'port': u''}], options=None) >Thread-26::DEBUG::2014-10-15 07:22:38,428::hsm::2372::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.66.128.104:_home_NFS1 >Thread-16::DEBUG::2014-10-15 07:22:38,428::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:38,428::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:38,428::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:38,428::__init__::232::IOProcess::(_processLogs) (8) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:22:38,436::__init__::232::IOProcess::(_processLogs) (8) Queuing response >Thread-26::DEBUG::2014-10-15 07:22:38,436::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: (u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4',) >Thread-26::DEBUG::2014-10-15 07:22:38,436::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4: storage.nfsSD.findDomain} >Thread-26::INFO::2014-10-15 07:22:38,436::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': u'cd8fa491-c53f-4601-8cc6-cc31a63494e7'}]} >Thread-26::DEBUG::2014-10-15 07:22:38,436::task::1191::Storage.TaskManager.Task::(prepare) Task=`b3b0d5df-c09e-48a7-888b-a0e4b8bd1b28`::finished: {'statuslist': [{'status': 0, 'id': u'cd8fa491-c53f-4601-8cc6-cc31a63494e7'}]} >Thread-26::DEBUG::2014-10-15 07:22:38,437::task::595::Storage.TaskManager.Task::(_updateState) Task=`b3b0d5df-c09e-48a7-888b-a0e4b8bd1b28`::moving from state preparing -> state finished >Thread-26::DEBUG::2014-10-15 07:22:38,437::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-26::DEBUG::2014-10-15 07:22:38,437::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-26::DEBUG::2014-10-15 07:22:38,437::task::993::Storage.TaskManager.Task::(_decref) Task=`b3b0d5df-c09e-48a7-888b-a0e4b8bd1b28`::ref 0 aborting False >Thread-26::DEBUG::2014-10-15 07:22:38,437::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connectStorageServer' in bridge with [{'status': 0, 'id': u'cd8fa491-c53f-4601-8cc6-cc31a63494e7'}] >Thread-26::DEBUG::2014-10-15 07:22:38,437::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:38,443::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:38,444::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-27::DEBUG::2014-10-15 07:22:38,444::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connect' in bridge with {u'masterVersion': 1, u'domainDict': {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': u'active'}, u'storagepoolID': u'00000002-0002-0002-0002-0000000003e1', u'scsiKey': u'00000002-0002-0002-0002-0000000003e1', u'masterSdUUID': u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', u'hostID': 2} >Thread-27::DEBUG::2014-10-15 07:22:38,444::task::595::Storage.TaskManager.Task::(_updateState) Task=`464b02d5-77da-42f3-b0d9-9571e456404a`::moving from state init -> state preparing >Thread-27::INFO::2014-10-15 07:22:38,445::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID=u'00000002-0002-0002-0002-0000000003e1', hostID=2, msdUUID=u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', masterVersion=1, domainsMap={u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': u'active'}, options=None) >Thread-27::DEBUG::2014-10-15 07:22:38,445::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`20bae2c6-2ca9-4f98-a201-0444e21bbeeb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-27::DEBUG::2014-10-15 07:22:38,445::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-27::DEBUG::2014-10-15 07:22:38,445::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-27::DEBUG::2014-10-15 07:22:38,445::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`20bae2c6-2ca9-4f98-a201-0444e21bbeeb`::Granted request >Thread-27::DEBUG::2014-10-15 07:22:38,446::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`38838481-162c-43c3-9c68-d69bcd2d452e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1059' at '_connectStoragePool' >Thread-27::DEBUG::2014-10-15 07:22:38,446::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-0000000003e1' for lock type 'exclusive' >Thread-27::DEBUG::2014-10-15 07:22:38,446::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free. Now locking as 'exclusive' (1 active user) >Thread-27::DEBUG::2014-10-15 07:22:38,446::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-0000000003e1`ReqID=`38838481-162c-43c3-9c68-d69bcd2d452e`::Granted request >Thread-27::DEBUG::2014-10-15 07:22:38,446::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 1 and domains map to {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': u'active'} >Thread-27::INFO::2014-10-15 07:22:38,446::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 1 and domains map {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': u'Active'} >Thread-27::INFO::2014-10-15 07:22:38,446::sp::133::Storage.StoragePool::(setBackend) updating pool 00000002-0002-0002-0002-0000000003e1 backend from type NoneType instance 0x7f3e17d3df00 to type StoragePoolMemoryBackend instance 0x7f3d9c3ec788 >Thread-27::INFO::2014-10-15 07:22:38,447::sp::631::Storage.StoragePool::(connect) Connect host #2 to the storage pool 00000002-0002-0002-0002-0000000003e1 with master domain: 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 (ver = 1) >Thread-27::DEBUG::2014-10-15 07:22:38,447::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-27::DEBUG::2014-10-15 07:22:38,447::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-27::DEBUG::2014-10-15 07:22:38,447::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-27::DEBUG::2014-10-15 07:22:38,447::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-27::DEBUG::2014-10-15 07:22:38,447::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-27::DEBUG::2014-10-15 07:22:38,447::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-27::DEBUG::2014-10-15 07:22:38,447::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-27::DEBUG::2014-10-15 07:22:38,447::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-27::DEBUG::2014-10-15 07:22:38,448::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-27::DEBUG::2014-10-15 07:22:38,448::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-27::DEBUG::2014-10-15 07:22:38,448::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-27::DEBUG::2014-10-15 07:22:38,448::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-27::DEBUG::2014-10-15 07:22:38,461::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-27::DEBUG::2014-10-15 07:22:38,462::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-27::DEBUG::2014-10-15 07:22:38,474::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-27::DEBUG::2014-10-15 07:22:38,475::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-27::DEBUG::2014-10-15 07:22:38,475::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-27::DEBUG::2014-10-15 07:22:38,475::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-27::DEBUG::2014-10-15 07:22:38,475::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-27::DEBUG::2014-10-15 07:22:38,475::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-27::DEBUG::2014-10-15 07:22:38,475::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-27::DEBUG::2014-10-15 07:22:38,475::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-16::DEBUG::2014-10-15 07:22:38,476::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:38,476::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:38,476::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:38,476::__init__::232::IOProcess::(_processLogs) (9) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:22:38,477::__init__::232::IOProcess::(_processLogs) (9) Queuing response >Thread-27::DEBUG::2014-10-15 07:22:38,484::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 07:22:38,484::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:38,484::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:38,485::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:38,485::__init__::232::IOProcess::(_processLogs) (10) Got request for method 'touch' >Thread-16::DEBUG::2014-10-15 07:22:38,526::__init__::232::IOProcess::(_processLogs) (10) Queuing response >Thread-27::DEBUG::2014-10-15 07:22:38,527::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-16::DEBUG::2014-10-15 07:22:38,527::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:38,527::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:38,527::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:38,527::__init__::232::IOProcess::(_processLogs) (11) Got request for method 'readfile' >Thread-16::DEBUG::2014-10-15 07:22:38,537::__init__::232::IOProcess::(_processLogs) (11) Queuing response >Thread-27::DEBUG::2014-10-15 07:22:38,537::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=domain1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-0000000003e1', 'REMOTE_PATH=10.66.128.104:/home/NFS1', 'ROLE=Master', 'SDUUID=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b90f33781d11f6e6b0b3945de3167c0719a458c1'] >Thread-16::DEBUG::2014-10-15 07:22:38,537::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:38,538::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:38,538::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:38,538::__init__::232::IOProcess::(_processLogs) (12) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:22:38,538::__init__::232::IOProcess::(_processLogs) (12) Queuing response >Thread-16::DEBUG::2014-10-15 07:22:38,538::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:38,538::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:38,539::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:38,539::__init__::232::IOProcess::(_processLogs) (13) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:22:38,539::__init__::232::IOProcess::(_processLogs) (13) Queuing response >Thread-27::DEBUG::2014-10-15 07:22:38,539::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-27::INFO::2014-10-15 07:22:38,539::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_imageNS already registered >Thread-27::INFO::2014-10-15 07:22:38,539::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_volumeNS already registered >Thread-27::DEBUG::2014-10-15 07:22:38,540::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 verified, version 1 >Thread-27::DEBUG::2014-10-15 07:22:38,540::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-27::DEBUG::2014-10-15 07:22:38,540::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-27::INFO::2014-10-15 07:22:38,540::domainMonitor::100::Storage.DomainMonitor::(startMonitoring) Start monitoring 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-27::DEBUG::2014-10-15 07:22:38,540::sp::1413::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `00000002-0002-0002-0002-0000000003e1` started monitoring domain `55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4` >Thread-29::DEBUG::2014-10-15 07:22:38,540::domainMonitor::182::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-27::DEBUG::2014-10-15 07:22:38,541::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-27::DEBUG::2014-10-15 07:22:38,541::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/00000002-0002-0002-0002-0000000003e1 >Thread-16::DEBUG::2014-10-15 07:22:38,542::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:38,542::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-29::DEBUG::2014-10-15 07:22:38,543::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-10-15 07:22:38,543::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:38,549::__init__::232::IOProcess::(_processLogs) (14) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:22:38,551::__init__::232::IOProcess::(_processLogs) (14) Queuing response >Thread-29::DEBUG::2014-10-15 07:22:38,551::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000351189 s, 1.3 MB/s\n'; <rc> = 0 >Thread-27::DEBUG::2014-10-15 07:22:38,551::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-16::DEBUG::2014-10-15 07:22:38,552::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:38,553::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:38,553::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:38,554::__init__::232::IOProcess::(_processLogs) (15) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:22:38,555::__init__::232::IOProcess::(_processLogs) (15) Queuing response >Thread-27::DEBUG::2014-10-15 07:22:38,556::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/ST31000524AS_5VPC8EVJ|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-16::DEBUG::2014-10-15 07:22:38,556::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:38,561::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:38,561::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:38,561::__init__::232::IOProcess::(_processLogs) (16) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:22:38,562::__init__::232::IOProcess::(_processLogs) (16) Queuing response >Thread-16::DEBUG::2014-10-15 07:22:38,562::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:38,562::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:38,562::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:38,562::__init__::232::IOProcess::(_processLogs) (17) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:22:38,563::__init__::232::IOProcess::(_processLogs) (17) Queuing response >Thread-16::DEBUG::2014-10-15 07:22:38,563::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:38,563::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:38,563::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:38,563::__init__::232::IOProcess::(_processLogs) (18) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:22:38,564::__init__::232::IOProcess::(_processLogs) (18) Queuing response >Thread-29::DEBUG::2014-10-15 07:22:38,564::domainMonitor::264::Storage.DomainMonitorThread::(_monitorDomain) Domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 changed its status to Valid >Thread-30::DEBUG::2014-10-15 07:22:38,564::misc::821::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-29::INFO::2014-10-15 07:22:38,565::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 (id: 2) >Thread-30::DEBUG::2014-10-15 07:22:38,565::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-30::DEBUG::2014-10-15 07:22:38,566::misc::841::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-29::DEBUG::2014-10-15 07:22:38,566::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 successfully acquired (id: 2) >Thread-31::INFO::2014-10-15 07:22:38,566::clientIF::136::vds::(contEIOVms) vmContainerLock acquired >Thread-27::DEBUG::2014-10-15 07:22:38,611::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-27::DEBUG::2014-10-15 07:22:38,612::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-16::DEBUG::2014-10-15 07:22:38,612::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:38,613::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:38,613::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:38,613::__init__::232::IOProcess::(_processLogs) (19) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:22:38,615::__init__::232::IOProcess::(_processLogs) (19) Queuing response >Thread-27::INFO::2014-10-15 07:22:38,616::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 to /rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-27::DEBUG::2014-10-15 07:22:38,616::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 to /rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-27::INFO::2014-10-15 07:22:38,616::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 to /rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd >Thread-27::DEBUG::2014-10-15 07:22:38,616::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 to /rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd >Thread-27::DEBUG::2014-10-15 07:22:38,623::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 5, waitTimeout: 3, maxTasks: 500 >Thread-27::DEBUG::2014-10-15 07:22:38,624::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /usr/bin/dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/outbox iflag=direct,fullblock bs=512 count=8 skip=16 (cwd None) >Thread-27::DEBUG::2014-10-15 07:22:38,639::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00286644 s, 1.4 MB/s\n'; <rc> = 0 >Thread-27::INFO::2014-10-15 07:22:38,639::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/usr/bin/dd', 'of=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=16'] >Thread-27::DEBUG::2014-10-15 07:22:38,640::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /usr/bin/dd of=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=16 (cwd None) >Thread-27::DEBUG::2014-10-15 07:22:38,749::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.103545 s, 39.6 kB/s\n'; <rc> = 0 >Thread-27::DEBUG::2014-10-15 07:22:38,750::storage_mailbox::207::Storage.Mailbox.HSM::(__init__) HSM_MailboxMonitor created for pool 00000002-0002-0002-0002-0000000003e1 >Thread-27::DEBUG::2014-10-15 07:22:38,750::sp::462::Storage.StoragePool::(__createMailboxMonitor) HSM mailbox ready for pool 00000002-0002-0002-0002-0000000003e1 on master domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-27::DEBUG::2014-10-15 07:22:38,750::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-0000000003e1' >Thread-27::DEBUG::2014-10-15 07:22:38,751::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-0000000003e1' (0 active users) >Thread-27::DEBUG::2014-10-15 07:22:38,751::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-0000000003e1' is free, finding out if anyone is waiting for it. >Thread-27::DEBUG::2014-10-15 07:22:38,751::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-0000000003e1', Clearing records. >Thread-27::DEBUG::2014-10-15 07:22:38,751::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-27::DEBUG::2014-10-15 07:22:38,751::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-27::DEBUG::2014-10-15 07:22:38,751::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-27::DEBUG::2014-10-15 07:22:38,751::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-27::INFO::2014-10-15 07:22:38,751::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-27::DEBUG::2014-10-15 07:22:38,752::task::1191::Storage.TaskManager.Task::(prepare) Task=`464b02d5-77da-42f3-b0d9-9571e456404a`::finished: True >Thread-27::DEBUG::2014-10-15 07:22:38,752::task::595::Storage.TaskManager.Task::(_updateState) Task=`464b02d5-77da-42f3-b0d9-9571e456404a`::moving from state preparing -> state finished >Thread-27::DEBUG::2014-10-15 07:22:38,752::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-27::DEBUG::2014-10-15 07:22:38,752::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-27::DEBUG::2014-10-15 07:22:38,752::task::993::Storage.TaskManager.Task::(_decref) Task=`464b02d5-77da-42f3-b0d9-9571e456404a`::ref 0 aborting False >Thread-27::DEBUG::2014-10-15 07:22:38,752::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connect' in bridge with True >Thread-27::DEBUG::2014-10-15 07:22:38,752::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:38,754::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:38,754::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-39::DEBUG::2014-10-15 07:22:38,756::task::595::Storage.TaskManager.Task::(_updateState) Task=`a063ae4f-857a-4d30-96d6-91062b71e2f1`::moving from state init -> state preparing >Thread-39::INFO::2014-10-15 07:22:38,756::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-39::INFO::2014-10-15 07:22:38,756::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000351189', 'lastCheck': '0.2', 'valid': True}} >Thread-39::DEBUG::2014-10-15 07:22:38,756::task::1191::Storage.TaskManager.Task::(prepare) Task=`a063ae4f-857a-4d30-96d6-91062b71e2f1`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000351189', 'lastCheck': '0.2', 'valid': True}} >Thread-39::DEBUG::2014-10-15 07:22:38,756::task::595::Storage.TaskManager.Task::(_updateState) Task=`a063ae4f-857a-4d30-96d6-91062b71e2f1`::moving from state preparing -> state finished >Thread-39::DEBUG::2014-10-15 07:22:38,756::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-39::DEBUG::2014-10-15 07:22:38,757::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-39::DEBUG::2014-10-15 07:22:38,757::task::993::Storage.TaskManager.Task::(_decref) Task=`a063ae4f-857a-4d30-96d6-91062b71e2f1`::ref 0 aborting False >Thread-39::ERROR::2014-10-15 07:22:38,758::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-39::DEBUG::2014-10-15 07:22:38,759::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:38,768::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-40::DEBUG::2014-10-15 07:22:38,768::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.setMOMPolicyParameters' in bridge with {u'key_value_store': {u'balloonEnabled': False, u'ksmEnabled': True}} >JsonRpcServer::DEBUG::2014-10-15 07:22:38,768::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-40::DEBUG::2014-10-15 07:22:38,822::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.setMOMPolicyParameters' in bridge with True >Thread-40::DEBUG::2014-10-15 07:22:38,823::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:41,911::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:41,911::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-41::DEBUG::2014-10-15 07:22:41,912::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:41,914::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:41,914::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-42::DEBUG::2014-10-15 07:22:41,915::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-42::DEBUG::2014-10-15 07:22:41,916::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >clientIFinit::DEBUG::2014-10-15 07:22:42,649::task::595::Storage.TaskManager.Task::(_updateState) Task=`16c87b56-99f4-463b-9e97-d0d12facdaac`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:22:42,649::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >clientIFinit::INFO::2014-10-15 07:22:42,649::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': [u'00000002-0002-0002-0002-0000000003e1']} >clientIFinit::DEBUG::2014-10-15 07:22:42,649::task::1191::Storage.TaskManager.Task::(prepare) Task=`16c87b56-99f4-463b-9e97-d0d12facdaac`::finished: {'poollist': [u'00000002-0002-0002-0002-0000000003e1']} >clientIFinit::DEBUG::2014-10-15 07:22:42,649::task::595::Storage.TaskManager.Task::(_updateState) Task=`16c87b56-99f4-463b-9e97-d0d12facdaac`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:22:42,650::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:22:42,650::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:22:42,650::task::993::Storage.TaskManager.Task::(_decref) Task=`16c87b56-99f4-463b-9e97-d0d12facdaac`::ref 0 aborting False >clientIFinit::WARNING::2014-10-15 07:22:42,650::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': 'unix', 'alias': 'channel0', 'deviceType': 'channel', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '1'}}' found >clientIFinit::WARNING::2014-10-15 07:22:42,650::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': 'unix', 'alias': 'channel1', 'deviceType': 'channel', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '2'}}' found >clientIFinit::WARNING::2014-10-15 07:22:42,650::vm::2051::vm.Vm::(buildConfDevices) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Unknown type found, device: '{'device': 'spicevmc', 'alias': 'channel2', 'deviceType': 'channel', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '3'}}' found >clientIFinit::DEBUG::2014-10-15 07:22:42,651::task::595::Storage.TaskManager.Task::(_updateState) Task=`c5c7a676-bb37-481a-8e02-5396f54fe0c3`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:22:42,651::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', volUUID='486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:22:42,651::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:42,651::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:42,652::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:42,652::__init__::232::IOProcess::(_processLogs) (20) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:22:42,652::__init__::232::IOProcess::(_processLogs) (20) Queuing response >Thread-16::DEBUG::2014-10-15 07:22:42,653::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:42,653::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >clientIFinit::INFO::2014-10-15 07:22:42,653::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-16::DEBUG::2014-10-15 07:22:42,654::__init__::232::IOProcess::(_processLogs) Extracting request information... >clientIFinit::DEBUG::2014-10-15 07:22:42,654::task::1191::Storage.TaskManager.Task::(prepare) Task=`c5c7a676-bb37-481a-8e02-5396f54fe0c3`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-16::DEBUG::2014-10-15 07:22:42,654::__init__::232::IOProcess::(_processLogs) (21) Got request for method 'stat' >clientIFinit::DEBUG::2014-10-15 07:22:42,654::task::595::Storage.TaskManager.Task::(_updateState) Task=`c5c7a676-bb37-481a-8e02-5396f54fe0c3`::moving from state preparing -> state finished >Thread-16::DEBUG::2014-10-15 07:22:42,655::__init__::232::IOProcess::(_processLogs) (21) Queuing response >clientIFinit::DEBUG::2014-10-15 07:22:42,655::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >clientIFinit::DEBUG::2014-10-15 07:22:42,655::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:22:42,655::task::993::Storage.TaskManager.Task::(_decref) Task=`c5c7a676-bb37-481a-8e02-5396f54fe0c3`::ref 0 aborting False >clientIFinit::INFO::2014-10-15 07:22:42,655::clientIF::331::vds::(prepareVolumePath) prepared volume path: >clientIFinit::DEBUG::2014-10-15 07:22:42,656::task::595::Storage.TaskManager.Task::(_updateState) Task=`f961a00e-df6e-4018-80d7-900b42a55871`::moving from state init -> state preparing >clientIFinit::INFO::2014-10-15 07:22:42,656::logUtils::44::dispatcher::(wrapper) Run and protect: prepareImage(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', leafUUID='486f9fc4-3108-49db-ae60-4dc85544e8eb') >clientIFinit::DEBUG::2014-10-15 07:22:42,656::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4`ReqID=`e067f0d6-8b73-4caf-8b82-9c214fad6263`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '3210' at 'prepareImage' >clientIFinit::DEBUG::2014-10-15 07:22:42,656::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' for lock type 'shared' >clientIFinit::DEBUG::2014-10-15 07:22:42,656::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' is free. Now locking as 'shared' (1 active user) >clientIFinit::DEBUG::2014-10-15 07:22:42,656::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4`ReqID=`e067f0d6-8b73-4caf-8b82-9c214fad6263`::Granted request >clientIFinit::DEBUG::2014-10-15 07:22:42,657::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f961a00e-df6e-4018-80d7-900b42a55871`::_resourcesAcquired: Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 (shared) >clientIFinit::DEBUG::2014-10-15 07:22:42,657::task::993::Storage.TaskManager.Task::(_decref) Task=`f961a00e-df6e-4018-80d7-900b42a55871`::ref 1 aborting False >Thread-16::DEBUG::2014-10-15 07:22:42,657::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:42,657::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:42,657::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:42,657::__init__::232::IOProcess::(_processLogs) (22) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:22:42,659::__init__::232::IOProcess::(_processLogs) (22) Queuing response >clientIFinit::DEBUG::2014-10-15 07:22:42,660::fileSD::499::Storage.StorageDomain::(activateVolumes) Fixing permissions on /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb >Thread-16::DEBUG::2014-10-15 07:22:42,660::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:42,660::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >clientIFinit::DEBUG::2014-10-15 07:22:42,660::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /var/run/vdsm/storage/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 07:22:42,661::__init__::232::IOProcess::(_processLogs) Extracting request information... >clientIFinit::WARNING::2014-10-15 07:22:42,661::fileUtils::149::Storage.fileUtils::(createdir) Dir /var/run/vdsm/storage/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 already exists >Thread-16::DEBUG::2014-10-15 07:22:42,661::__init__::232::IOProcess::(_processLogs) (23) Got request for method 'stat' >clientIFinit::DEBUG::2014-10-15 07:22:42,661::fileSD::474::Storage.StorageDomain::(createImageLinks) Creating symlink from /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610 to /var/run/vdsm/storage/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/a5611808-d282-45f9-9771-c3797b521610 >Thread-16::DEBUG::2014-10-15 07:22:42,662::__init__::232::IOProcess::(_processLogs) (23) Queuing response >clientIFinit::DEBUG::2014-10-15 07:22:42,662::fileSD::479::Storage.StorageDomain::(createImageLinks) img run dir already exists: /var/run/vdsm/storage/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/a5611808-d282-45f9-9771-c3797b521610 >Thread-16::DEBUG::2014-10-15 07:22:42,663::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:42,663::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:42,663::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:42,663::__init__::232::IOProcess::(_processLogs) (24) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:22:42,663::__init__::232::IOProcess::(_processLogs) (24) Queuing response >Thread-16::DEBUG::2014-10-15 07:22:42,664::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:42,664::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >clientIFinit::DEBUG::2014-10-15 07:22:42,664::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for 486f9fc4-3108-49db-ae60-4dc85544e8eb >Thread-16::DEBUG::2014-10-15 07:22:42,664::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:42,665::__init__::232::IOProcess::(_processLogs) (25) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:22:42,665::__init__::232::IOProcess::(_processLogs) (25) Queuing response >Thread-16::DEBUG::2014-10-15 07:22:42,665::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:42,665::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:42,665::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:42,665::__init__::232::IOProcess::(_processLogs) (26) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:22:42,665::__init__::232::IOProcess::(_processLogs) (26) Queuing response >Thread-16::DEBUG::2014-10-15 07:22:42,666::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:42,666::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:42,666::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:42,666::__init__::232::IOProcess::(_processLogs) (27) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:22:42,667::__init__::232::IOProcess::(_processLogs) (27) Queuing response >clientIFinit::INFO::2014-10-15 07:22:42,667::logUtils::47::dispatcher::(wrapper) Run and protect: prepareImage, Return response: {'info': {'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610'}, 'path': u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'imgVolumesInfo': [{'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610'}]} >clientIFinit::DEBUG::2014-10-15 07:22:42,667::task::1191::Storage.TaskManager.Task::(prepare) Task=`f961a00e-df6e-4018-80d7-900b42a55871`::finished: {'info': {'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610'}, 'path': u'/rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'imgVolumesInfo': [{'domainID': '55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb', 'volumeID': u'486f9fc4-3108-49db-ae60-4dc85544e8eb', 'leasePath': u'/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb.lease', 'imageID': 'a5611808-d282-45f9-9771-c3797b521610'}]} >clientIFinit::DEBUG::2014-10-15 07:22:42,667::task::595::Storage.TaskManager.Task::(_updateState) Task=`f961a00e-df6e-4018-80d7-900b42a55871`::moving from state preparing -> state finished >clientIFinit::DEBUG::2014-10-15 07:22:42,667::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': < ResourceRef 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', isValid: 'True' obj: 'None'>} >clientIFinit::DEBUG::2014-10-15 07:22:42,667::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >clientIFinit::DEBUG::2014-10-15 07:22:42,667::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' >clientIFinit::DEBUG::2014-10-15 07:22:42,668::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' (0 active users) >clientIFinit::DEBUG::2014-10-15 07:22:42,668::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' is free, finding out if anyone is waiting for it. >clientIFinit::DEBUG::2014-10-15 07:22:42,668::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', Clearing records. >clientIFinit::DEBUG::2014-10-15 07:22:42,668::task::993::Storage.TaskManager.Task::(_decref) Task=`f961a00e-df6e-4018-80d7-900b42a55871`::ref 0 aborting False >clientIFinit::INFO::2014-10-15 07:22:42,668::clientIF::331::vds::(prepareVolumePath) prepared volume path: /rhev/data-center/00000002-0002-0002-0002-0000000003e1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/images/a5611808-d282-45f9-9771-c3797b521610/486f9fc4-3108-49db-ae60-4dc85544e8eb >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:44,937::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:44,937::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-43::DEBUG::2014-10-15 07:22:44,939::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:44,940::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:44,940::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-44::DEBUG::2014-10-15 07:22:44,941::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-44::DEBUG::2014-10-15 07:22:44,941::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-21::INFO::2014-10-15 07:22:45,529::migration::466::vm.Vm::(monitor_migration) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Migration Progress: 10 seconds elapsed, 34% of data processed >Thread-17::DEBUG::2014-10-15 07:22:46,916::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c0d4b45-99fb-400c-a81a-e617c77530d4`::moving from state init -> state preparing >Thread-17::INFO::2014-10-15 07:22:46,916::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', volUUID='486f9fc4-3108-49db-ae60-4dc85544e8eb', options=None) >Thread-16::DEBUG::2014-10-15 07:22:46,917::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:46,917::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:46,918::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:46,918::__init__::232::IOProcess::(_processLogs) (28) Got request for method 'stat' >Thread-16::DEBUG::2014-10-15 07:22:46,918::__init__::232::IOProcess::(_processLogs) (28) Queuing response >Thread-16::DEBUG::2014-10-15 07:22:46,918::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:46,918::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-17::INFO::2014-10-15 07:22:46,918::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-16::DEBUG::2014-10-15 07:22:46,919::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-17::DEBUG::2014-10-15 07:22:46,919::task::1191::Storage.TaskManager.Task::(prepare) Task=`1c0d4b45-99fb-400c-a81a-e617c77530d4`::finished: {'truesize': '4116598784', 'apparentsize': '10737418240'} >Thread-17::DEBUG::2014-10-15 07:22:46,919::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c0d4b45-99fb-400c-a81a-e617c77530d4`::moving from state preparing -> state finished >Thread-16::DEBUG::2014-10-15 07:22:46,919::__init__::232::IOProcess::(_processLogs) (29) Got request for method 'stat' >Thread-17::DEBUG::2014-10-15 07:22:46,920::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2014-10-15 07:22:46,920::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-16::DEBUG::2014-10-15 07:22:46,920::__init__::232::IOProcess::(_processLogs) (29) Queuing response >Thread-17::DEBUG::2014-10-15 07:22:46,921::task::993::Storage.TaskManager.Task::(_decref) Task=`1c0d4b45-99fb-400c-a81a-e617c77530d4`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-10-15 07:22:47,477::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 33. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:47,957::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:47,958::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-45::DEBUG::2014-10-15 07:22:47,959::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:47,961::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:47,961::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-46::DEBUG::2014-10-15 07:22:47,962::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-46::DEBUG::2014-10-15 07:22:47,962::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:22:48,566::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:48,567::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:48,567::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:48,567::__init__::232::IOProcess::(_processLogs) (30) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:22:48,567::__init__::232::IOProcess::(_processLogs) (30) Queuing response >Thread-29::DEBUG::2014-10-15 07:22:48,567::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:22:48,574::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000403563 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:22:48,575::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:48,575::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:48,575::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:48,575::__init__::232::IOProcess::(_processLogs) (31) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:22:48,575::__init__::232::IOProcess::(_processLogs) (31) Queuing response >Thread-16::DEBUG::2014-10-15 07:22:48,576::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:48,576::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:48,577::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:48,577::__init__::232::IOProcess::(_processLogs) (32) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:22:48,577::__init__::232::IOProcess::(_processLogs) (32) Queuing response >Thread-16::DEBUG::2014-10-15 07:22:48,577::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:48,577::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:48,578::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:48,578::__init__::232::IOProcess::(_processLogs) (33) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:22:48,578::__init__::232::IOProcess::(_processLogs) (33) Queuing response >Thread-16::DEBUG::2014-10-15 07:22:48,578::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:48,578::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:48,578::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:48,578::__init__::232::IOProcess::(_processLogs) (34) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:22:48,578::__init__::232::IOProcess::(_processLogs) (34) Queuing response >Thread-20::DEBUG::2014-10-15 07:22:50,529::migration::369::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::setting migration downtime to 50 >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:50,987::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:50,988::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-47::DEBUG::2014-10-15 07:22:50,989::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f902b9c-569b-4c74-ba05-a8475ea88dd7`::moving from state init -> state preparing >Thread-47::INFO::2014-10-15 07:22:50,989::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-10-15 07:22:50,990::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000403563', 'lastCheck': '2.4', 'valid': True}} >Thread-47::DEBUG::2014-10-15 07:22:50,990::task::1191::Storage.TaskManager.Task::(prepare) Task=`9f902b9c-569b-4c74-ba05-a8475ea88dd7`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000403563', 'lastCheck': '2.4', 'valid': True}} >Thread-47::DEBUG::2014-10-15 07:22:50,990::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f902b9c-569b-4c74-ba05-a8475ea88dd7`::moving from state preparing -> state finished >Thread-47::DEBUG::2014-10-15 07:22:50,990::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2014-10-15 07:22:50,990::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2014-10-15 07:22:50,990::task::993::Storage.TaskManager.Task::(_decref) Task=`9f902b9c-569b-4c74-ba05-a8475ea88dd7`::ref 0 aborting False >Thread-47::ERROR::2014-10-15 07:22:50,991::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-47::DEBUG::2014-10-15 07:22:50,992::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:51,003::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:51,004::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-48::DEBUG::2014-10-15 07:22:51,005::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-48::DEBUG::2014-10-15 07:22:51,005::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:54,113::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:54,114::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-49::DEBUG::2014-10-15 07:22:54,115::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:54,117::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:54,118::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-50::DEBUG::2014-10-15 07:22:54,119::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-50::DEBUG::2014-10-15 07:22:54,119::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-21::INFO::2014-10-15 07:22:55,531::migration::466::vm.Vm::(monitor_migration) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Migration Progress: 20 seconds elapsed, 85% of data processed >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:57,153::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:57,154::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-51::DEBUG::2014-10-15 07:22:57,155::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:22:57,156::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:22:57,157::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-52::DEBUG::2014-10-15 07:22:57,157::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-52::DEBUG::2014-10-15 07:22:57,158::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:22:58,579::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:58,580::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:58,580::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:58,580::__init__::232::IOProcess::(_processLogs) (35) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:22:58,580::__init__::232::IOProcess::(_processLogs) (35) Queuing response >Thread-29::DEBUG::2014-10-15 07:22:58,581::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:22:58,587::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000358004 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:22:58,588::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:58,588::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:58,588::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:58,589::__init__::232::IOProcess::(_processLogs) (36) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:22:58,589::__init__::232::IOProcess::(_processLogs) (36) Queuing response >Thread-16::DEBUG::2014-10-15 07:22:58,589::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:58,590::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:58,590::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:58,590::__init__::232::IOProcess::(_processLogs) (37) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:22:58,590::__init__::232::IOProcess::(_processLogs) (37) Queuing response >Thread-16::DEBUG::2014-10-15 07:22:58,590::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:58,590::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:58,590::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:58,591::__init__::232::IOProcess::(_processLogs) (38) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:22:58,591::__init__::232::IOProcess::(_processLogs) (38) Queuing response >Thread-16::DEBUG::2014-10-15 07:22:58,591::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:22:58,591::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:22:58,591::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:22:58,592::__init__::232::IOProcess::(_processLogs) (39) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:22:58,592::__init__::232::IOProcess::(_processLogs) (39) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:00,178::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:00,178::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-53::DEBUG::2014-10-15 07:23:00,179::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:00,181::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:00,181::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-54::DEBUG::2014-10-15 07:23:00,182::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Domain Metadata is not set >Thread-54::DEBUG::2014-10-15 07:23:00,182::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >libvirtEventLoop::DEBUG::2014-10-15 07:23:02,577::vm::5461::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::event Suspended detail 0 opaque None >VM Channels Listener::ERROR::2014-10-15 07:23:02,940::vmchannels::54::vds::(_handle_event) Received 00000019 on fileno 33 >VM Channels Listener::DEBUG::2014-10-15 07:23:02,941::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 33. >VM Channels Listener::DEBUG::2014-10-15 07:23:02,941::guestagent::200::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Attempting connection to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-10-15 07:23:02,942::guestagent::214::vm.Vm::(_connect) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Failed to connect to /var/lib/libvirt/qemu/channels/c255610f-42c2-426d-9270-e4497f4a0f37.com.redhat.rhevm.vdsm with 111 >libvirtEventLoop::DEBUG::2014-10-15 07:23:03,126::vm::5461::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::event Stopped detail 3 opaque None >Thread-19::DEBUG::2014-10-15 07:23:03,126::migration::375::vm.Vm::(cancel) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::canceling migration downtime thread >Thread-19::DEBUG::2014-10-15 07:23:03,127::migration::469::vm.Vm::(stop) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::stopping migration monitor thread >Thread-20::DEBUG::2014-10-15 07:23:03,127::migration::372::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::migration downtime thread exiting >Thread-21::DEBUG::2014-10-15 07:23:03,128::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 42 edom: 10 level: 2 message: Domain not found: no domain with matching uuid 'c255610f-42c2-426d-9270-e4497f4a0f37' (rhevh-guest-rhel7.0) >Thread-19::DEBUG::2014-10-15 07:23:03,131::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 42 edom: 10 level: 2 message: Domain not found: no domain with matching uuid 'c255610f-42c2-426d-9270-e4497f4a0f37' (rhevh-guest-rhel7.0) >Thread-19::DEBUG::2014-10-15 07:23:03,131::vm::2838::vm.Vm::(setDownStatus) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Changed state to Down: Migration succeeded (code=4) >Thread-19::DEBUG::2014-10-15 07:23:03,131::vmchannels::210::vds::(unregister) Delete fileno 33 from listener. >Thread-19::DEBUG::2014-10-15 07:23:03,132::sampling::412::vm.Vm::(stop) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stop statistics collection >Thread-17::DEBUG::2014-10-15 07:23:03,132::sampling::443::vm.Vm::(run) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stats thread finished >Thread-19::DEBUG::2014-10-15 07:23:03,134::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 42 edom: 10 level: 2 message: Domain not found: no domain with matching uuid 'c255610f-42c2-426d-9270-e4497f4a0f37' (rhevh-guest-rhel7.0) >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:03,211::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:03,212::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-55::DEBUG::2014-10-15 07:23:03,213::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:03,214::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:03,215::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-56::DEBUG::2014-10-15 07:23:03,215::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:03,221::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:03,221::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-57::DEBUG::2014-10-15 07:23:03,221::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'VM.destroy' in bridge with {u'vmID': u'c255610f-42c2-426d-9270-e4497f4a0f37'} >Thread-57::INFO::2014-10-15 07:23:03,222::API::332::vds::(destroy) vmContainerLock acquired by vm c255610f-42c2-426d-9270-e4497f4a0f37 >Thread-57::DEBUG::2014-10-15 07:23:03,222::vm::4919::vm.Vm::(destroy) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::destroy Called >Thread-57::INFO::2014-10-15 07:23:03,222::vm::4845::vm.Vm::(releaseVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Release VM resources >Thread-57::WARNING::2014-10-15 07:23:03,222::vm::1955::vm.Vm::(_set_lastStatus) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::trying to set state to Powering down when already Down >Thread-57::DEBUG::2014-10-15 07:23:03,223::sampling::412::vm.Vm::(stop) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Stop statistics collection >Thread-57::DEBUG::2014-10-15 07:23:03,223::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 42 edom: 10 level: 2 message: Domain not found: no domain with matching uuid 'c255610f-42c2-426d-9270-e4497f4a0f37' (rhevh-guest-rhel7.0) >Thread-57::INFO::2014-10-15 07:23:03,223::vm::4888::vm.Vm::(_destroyVmGraceful) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::VM 'c255610f-42c2-426d-9270-e4497f4a0f37' already down and destroyed >Thread-57::DEBUG::2014-10-15 07:23:03,224::task::595::Storage.TaskManager.Task::(_updateState) Task=`64b0f562-951a-4161-9fca-5f1075abd5f8`::moving from state init -> state preparing >Thread-57::INFO::2014-10-15 07:23:03,224::logUtils::44::dispatcher::(wrapper) Run and protect: teardownImage(sdUUID='55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', spUUID='00000002-0002-0002-0002-0000000003e1', imgUUID='a5611808-d282-45f9-9771-c3797b521610', volUUID=None) >Thread-57::DEBUG::2014-10-15 07:23:03,224::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4`ReqID=`aef23c1f-533d-4b54-b002-7dbe9e6e8626`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '3262' at 'teardownImage' >Thread-57::DEBUG::2014-10-15 07:23:03,224::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' for lock type 'shared' >Thread-57::DEBUG::2014-10-15 07:23:03,224::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' is free. Now locking as 'shared' (1 active user) >Thread-57::DEBUG::2014-10-15 07:23:03,224::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4`ReqID=`aef23c1f-533d-4b54-b002-7dbe9e6e8626`::Granted request >Thread-57::DEBUG::2014-10-15 07:23:03,225::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`64b0f562-951a-4161-9fca-5f1075abd5f8`::_resourcesAcquired: Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 (shared) >Thread-57::DEBUG::2014-10-15 07:23:03,225::task::993::Storage.TaskManager.Task::(_decref) Task=`64b0f562-951a-4161-9fca-5f1075abd5f8`::ref 1 aborting False >Thread-57::INFO::2014-10-15 07:23:03,225::logUtils::47::dispatcher::(wrapper) Run and protect: teardownImage, Return response: None >Thread-57::DEBUG::2014-10-15 07:23:03,225::task::1191::Storage.TaskManager.Task::(prepare) Task=`64b0f562-951a-4161-9fca-5f1075abd5f8`::finished: None >Thread-57::DEBUG::2014-10-15 07:23:03,225::task::595::Storage.TaskManager.Task::(_updateState) Task=`64b0f562-951a-4161-9fca-5f1075abd5f8`::moving from state preparing -> state finished >Thread-57::DEBUG::2014-10-15 07:23:03,225::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': < ResourceRef 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', isValid: 'True' obj: 'None'>} >Thread-57::DEBUG::2014-10-15 07:23:03,225::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-57::DEBUG::2014-10-15 07:23:03,225::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' >Thread-57::DEBUG::2014-10-15 07:23:03,225::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' (0 active users) >Thread-57::DEBUG::2014-10-15 07:23:03,225::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4' is free, finding out if anyone is waiting for it. >Thread-57::DEBUG::2014-10-15 07:23:03,226::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', Clearing records. >Thread-57::DEBUG::2014-10-15 07:23:03,226::task::993::Storage.TaskManager.Task::(_decref) Task=`64b0f562-951a-4161-9fca-5f1075abd5f8`::ref 0 aborting False >Thread-57::DEBUG::2014-10-15 07:23:03,226::task::595::Storage.TaskManager.Task::(_updateState) Task=`33055b34-bb37-4523-8201-c22d777c9e0d`::moving from state init -> state preparing >Thread-57::INFO::2014-10-15 07:23:03,226::logUtils::44::dispatcher::(wrapper) Run and protect: inappropriateDevices(thiefId='c255610f-42c2-426d-9270-e4497f4a0f37') >Thread-57::INFO::2014-10-15 07:23:03,227::logUtils::47::dispatcher::(wrapper) Run and protect: inappropriateDevices, Return response: None >Thread-57::DEBUG::2014-10-15 07:23:03,227::task::1191::Storage.TaskManager.Task::(prepare) Task=`33055b34-bb37-4523-8201-c22d777c9e0d`::finished: None >Thread-57::DEBUG::2014-10-15 07:23:03,227::task::595::Storage.TaskManager.Task::(_updateState) Task=`33055b34-bb37-4523-8201-c22d777c9e0d`::moving from state preparing -> state finished >Thread-57::DEBUG::2014-10-15 07:23:03,228::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-57::DEBUG::2014-10-15 07:23:03,228::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-57::DEBUG::2014-10-15 07:23:03,228::task::993::Storage.TaskManager.Task::(_decref) Task=`33055b34-bb37-4523-8201-c22d777c9e0d`::ref 0 aborting False >Thread-57::DEBUG::2014-10-15 07:23:03,228::utils::738::root::(execCmd) /usr/libexec/vdsm/hooks/after_vm_destroy/50_vhostmd (cwd None) >Detector thread::DEBUG::2014-10-15 07:23:03,313::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 127.0.0.1:57014 >Detector thread::DEBUG::2014-10-15 07:23:03,314::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 127.0.0.1:57014 >Detector thread::DEBUG::2014-10-15 07:23:03,314::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 127.0.0.1:57014 >Detector thread::DEBUG::2014-10-15 07:23:03,314::BindingXMLRPC::1172::XmlDetector::(handleSocket) xml over http detected from ('127.0.0.1', 57014) >Thread-57::DEBUG::2014-10-15 07:23:03,362::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::INFO::2014-10-15 07:23:03,362::hooks::100::root::(_runHooksDir) >Thread-57::DEBUG::2014-10-15 07:23:03,363::vm::4913::vm.Vm::(deleteVm) vmId=`c255610f-42c2-426d-9270-e4497f4a0f37`::Total desktops after destroy of c255610f-42c2-426d-9270-e4497f4a0f37 is 0 >Thread-57::DEBUG::2014-10-15 07:23:03,363::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'VM.destroy' in bridge with True >Thread-57::DEBUG::2014-10-15 07:23:03,363::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:03,368::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-59::DEBUG::2014-10-15 07:23:03,369::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getVMFullList' in bridge with {u'vmList': [u'c255610f-42c2-426d-9270-e4497f4a0f37']} >JsonRpcServer::DEBUG::2014-10-15 07:23:03,369::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-59::DEBUG::2014-10-15 07:23:03,370::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getVMFullList' in bridge with [] >Thread-59::DEBUG::2014-10-15 07:23:03,370::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >VM Channels Listener::DEBUG::2014-10-15 07:23:03,943::vmchannels::117::vds::(_do_del_channels) fileno 33 was removed from listener. >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:06,422::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:06,422::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-60::DEBUG::2014-10-15 07:23:06,424::task::595::Storage.TaskManager.Task::(_updateState) Task=`bef958d3-45e3-4bdb-8fbc-7f004d1cb548`::moving from state init -> state preparing >Thread-60::INFO::2014-10-15 07:23:06,424::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-60::INFO::2014-10-15 07:23:06,424::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000358004', 'lastCheck': '7.8', 'valid': True}} >Thread-60::DEBUG::2014-10-15 07:23:06,424::task::1191::Storage.TaskManager.Task::(prepare) Task=`bef958d3-45e3-4bdb-8fbc-7f004d1cb548`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000358004', 'lastCheck': '7.8', 'valid': True}} >Thread-60::DEBUG::2014-10-15 07:23:06,424::task::595::Storage.TaskManager.Task::(_updateState) Task=`bef958d3-45e3-4bdb-8fbc-7f004d1cb548`::moving from state preparing -> state finished >Thread-60::DEBUG::2014-10-15 07:23:06,424::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-60::DEBUG::2014-10-15 07:23:06,424::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-60::DEBUG::2014-10-15 07:23:06,424::task::993::Storage.TaskManager.Task::(_decref) Task=`bef958d3-45e3-4bdb-8fbc-7f004d1cb548`::ref 0 aborting False >Thread-60::ERROR::2014-10-15 07:23:06,425::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-60::DEBUG::2014-10-15 07:23:06,426::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:06,437::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:06,438::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-61::DEBUG::2014-10-15 07:23:06,439::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:23:08,592::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:08,593::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:08,593::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:08,593::__init__::232::IOProcess::(_processLogs) (40) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:23:08,593::__init__::232::IOProcess::(_processLogs) (40) Queuing response >Thread-29::DEBUG::2014-10-15 07:23:08,594::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:23:08,600::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000369889 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:23:08,601::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:08,601::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:08,601::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:08,601::__init__::232::IOProcess::(_processLogs) (41) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:23:08,602::__init__::232::IOProcess::(_processLogs) (41) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:08,602::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:08,603::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:08,603::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:08,603::__init__::232::IOProcess::(_processLogs) (42) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:08,603::__init__::232::IOProcess::(_processLogs) (42) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:08,603::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:08,603::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:08,604::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:08,604::__init__::232::IOProcess::(_processLogs) (43) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:08,604::__init__::232::IOProcess::(_processLogs) (43) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:08,604::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:08,604::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:08,604::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:08,605::__init__::232::IOProcess::(_processLogs) (44) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:08,605::__init__::232::IOProcess::(_processLogs) (44) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:09,503::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:09,504::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-62::DEBUG::2014-10-15 07:23:09,505::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:12,516::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:12,517::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-63::DEBUG::2014-10-15 07:23:12,518::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:15,528::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:15,529::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-64::DEBUG::2014-10-15 07:23:15,531::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:18,538::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:18,539::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-65::DEBUG::2014-10-15 07:23:18,541::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:23:18,606::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:18,606::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:18,606::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:18,606::__init__::232::IOProcess::(_processLogs) (45) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:23:18,606::__init__::232::IOProcess::(_processLogs) (45) Queuing response >Thread-29::DEBUG::2014-10-15 07:23:18,607::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:23:18,613::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000373573 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:23:18,614::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:18,614::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:18,614::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:18,614::__init__::232::IOProcess::(_processLogs) (46) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:23:18,615::__init__::232::IOProcess::(_processLogs) (46) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:18,615::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:18,616::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:18,616::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:18,616::__init__::232::IOProcess::(_processLogs) (47) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:18,616::__init__::232::IOProcess::(_processLogs) (47) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:18,616::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:18,616::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:18,617::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:18,617::__init__::232::IOProcess::(_processLogs) (48) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:18,617::__init__::232::IOProcess::(_processLogs) (48) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:18,617::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:18,617::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:18,617::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:18,618::__init__::232::IOProcess::(_processLogs) (49) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:18,618::__init__::232::IOProcess::(_processLogs) (49) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:21,546::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:21,547::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-66::DEBUG::2014-10-15 07:23:21,548::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ddf3c37-60fa-4190-828a-5479988e25b8`::moving from state init -> state preparing >Thread-66::INFO::2014-10-15 07:23:21,548::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-66::INFO::2014-10-15 07:23:21,548::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000373573', 'lastCheck': '2.9', 'valid': True}} >Thread-66::DEBUG::2014-10-15 07:23:21,548::task::1191::Storage.TaskManager.Task::(prepare) Task=`0ddf3c37-60fa-4190-828a-5479988e25b8`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000373573', 'lastCheck': '2.9', 'valid': True}} >Thread-66::DEBUG::2014-10-15 07:23:21,549::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ddf3c37-60fa-4190-828a-5479988e25b8`::moving from state preparing -> state finished >Thread-66::DEBUG::2014-10-15 07:23:21,549::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-66::DEBUG::2014-10-15 07:23:21,549::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2014-10-15 07:23:21,549::task::993::Storage.TaskManager.Task::(_decref) Task=`0ddf3c37-60fa-4190-828a-5479988e25b8`::ref 0 aborting False >Thread-66::ERROR::2014-10-15 07:23:21,550::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-66::DEBUG::2014-10-15 07:23:21,550::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:21,557::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:21,558::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-67::DEBUG::2014-10-15 07:23:21,559::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:24,632::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:24,633::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-68::DEBUG::2014-10-15 07:23:24,634::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:27,642::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:27,643::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-69::DEBUG::2014-10-15 07:23:27,644::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:23:28,618::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:28,619::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:28,619::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:28,619::__init__::232::IOProcess::(_processLogs) (50) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:23:28,619::__init__::232::IOProcess::(_processLogs) (50) Queuing response >Thread-29::DEBUG::2014-10-15 07:23:28,620::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:23:28,626::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000397496 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:23:28,627::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:28,627::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:28,627::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:28,627::__init__::232::IOProcess::(_processLogs) (51) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:23:28,628::__init__::232::IOProcess::(_processLogs) (51) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:28,628::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:28,628::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:28,629::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:28,629::__init__::232::IOProcess::(_processLogs) (52) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:28,629::__init__::232::IOProcess::(_processLogs) (52) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:28,629::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:28,629::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:28,629::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:28,630::__init__::232::IOProcess::(_processLogs) (53) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:28,630::__init__::232::IOProcess::(_processLogs) (53) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:28,630::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:28,630::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:28,630::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:28,631::__init__::232::IOProcess::(_processLogs) (54) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:28,631::__init__::232::IOProcess::(_processLogs) (54) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:30,652::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:30,653::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-70::DEBUG::2014-10-15 07:23:30,654::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:33,664::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:33,665::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-71::DEBUG::2014-10-15 07:23:33,666::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:36,676::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:36,676::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-72::DEBUG::2014-10-15 07:23:36,677::task::595::Storage.TaskManager.Task::(_updateState) Task=`a01a2300-4695-4cda-923a-93f54ce9788d`::moving from state init -> state preparing >Thread-72::INFO::2014-10-15 07:23:36,677::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-72::INFO::2014-10-15 07:23:36,678::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000397496', 'lastCheck': '8.0', 'valid': True}} >Thread-72::DEBUG::2014-10-15 07:23:36,678::task::1191::Storage.TaskManager.Task::(prepare) Task=`a01a2300-4695-4cda-923a-93f54ce9788d`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000397496', 'lastCheck': '8.0', 'valid': True}} >Thread-72::DEBUG::2014-10-15 07:23:36,678::task::595::Storage.TaskManager.Task::(_updateState) Task=`a01a2300-4695-4cda-923a-93f54ce9788d`::moving from state preparing -> state finished >Thread-72::DEBUG::2014-10-15 07:23:36,678::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-72::DEBUG::2014-10-15 07:23:36,678::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-72::DEBUG::2014-10-15 07:23:36,678::task::993::Storage.TaskManager.Task::(_decref) Task=`a01a2300-4695-4cda-923a-93f54ce9788d`::ref 0 aborting False >Thread-72::ERROR::2014-10-15 07:23:36,679::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-72::DEBUG::2014-10-15 07:23:36,679::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:36,690::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:36,691::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-73::DEBUG::2014-10-15 07:23:36,691::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:23:38,631::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:38,632::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:38,632::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:38,632::__init__::232::IOProcess::(_processLogs) (55) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:23:38,632::__init__::232::IOProcess::(_processLogs) (55) Queuing response >Thread-29::DEBUG::2014-10-15 07:23:38,633::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:23:38,641::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000452786 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:23:38,641::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:38,641::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:38,642::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:38,642::__init__::232::IOProcess::(_processLogs) (56) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:23:38,642::__init__::232::IOProcess::(_processLogs) (56) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:38,643::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:38,643::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:38,643::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:38,643::__init__::232::IOProcess::(_processLogs) (57) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:38,643::__init__::232::IOProcess::(_processLogs) (57) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:38,643::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:38,644::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:38,644::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:38,644::__init__::232::IOProcess::(_processLogs) (58) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:38,644::__init__::232::IOProcess::(_processLogs) (58) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:38,644::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:38,644::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:38,645::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:38,645::__init__::232::IOProcess::(_processLogs) (59) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:38,645::__init__::232::IOProcess::(_processLogs) (59) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:39,764::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:39,765::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-74::DEBUG::2014-10-15 07:23:39,766::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:42,774::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:42,775::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-75::DEBUG::2014-10-15 07:23:42,776::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:45,788::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:45,789::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-76::DEBUG::2014-10-15 07:23:45,792::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:23:48,646::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:48,646::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:48,646::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:48,646::__init__::232::IOProcess::(_processLogs) (60) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:23:48,647::__init__::232::IOProcess::(_processLogs) (60) Queuing response >Thread-29::DEBUG::2014-10-15 07:23:48,647::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:23:48,654::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000392883 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:23:48,655::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:48,655::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:48,655::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:48,655::__init__::232::IOProcess::(_processLogs) (61) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:23:48,655::__init__::232::IOProcess::(_processLogs) (61) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:48,656::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:48,656::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:48,657::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:48,657::__init__::232::IOProcess::(_processLogs) (62) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:48,657::__init__::232::IOProcess::(_processLogs) (62) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:48,657::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:48,657::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:48,658::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:48,658::__init__::232::IOProcess::(_processLogs) (63) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:48,658::__init__::232::IOProcess::(_processLogs) (63) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:48,658::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:48,658::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:48,658::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:48,659::__init__::232::IOProcess::(_processLogs) (64) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:48,659::__init__::232::IOProcess::(_processLogs) (64) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:48,804::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:48,805::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-77::DEBUG::2014-10-15 07:23:48,806::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:51,815::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:51,816::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-78::DEBUG::2014-10-15 07:23:51,817::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5da5c53-95d4-4094-bfcc-39a1b049f891`::moving from state init -> state preparing >Thread-78::INFO::2014-10-15 07:23:51,817::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-78::INFO::2014-10-15 07:23:51,818::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000392883', 'lastCheck': '3.2', 'valid': True}} >Thread-78::DEBUG::2014-10-15 07:23:51,818::task::1191::Storage.TaskManager.Task::(prepare) Task=`d5da5c53-95d4-4094-bfcc-39a1b049f891`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000392883', 'lastCheck': '3.2', 'valid': True}} >Thread-78::DEBUG::2014-10-15 07:23:51,818::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5da5c53-95d4-4094-bfcc-39a1b049f891`::moving from state preparing -> state finished >Thread-78::DEBUG::2014-10-15 07:23:51,818::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-78::DEBUG::2014-10-15 07:23:51,818::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-78::DEBUG::2014-10-15 07:23:51,818::task::993::Storage.TaskManager.Task::(_decref) Task=`d5da5c53-95d4-4094-bfcc-39a1b049f891`::ref 0 aborting False >Thread-78::ERROR::2014-10-15 07:23:51,819::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-78::DEBUG::2014-10-15 07:23:51,820::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:51,831::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:51,831::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-79::DEBUG::2014-10-15 07:23:51,832::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:54,900::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:54,901::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-80::DEBUG::2014-10-15 07:23:54,902::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:23:57,910::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:23:57,911::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-81::DEBUG::2014-10-15 07:23:57,912::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:23:58,659::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:58,660::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:58,660::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:58,660::__init__::232::IOProcess::(_processLogs) (65) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:23:58,660::__init__::232::IOProcess::(_processLogs) (65) Queuing response >Thread-29::DEBUG::2014-10-15 07:23:58,660::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:23:58,667::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00048748 s, 942 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:23:58,668::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:58,668::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:58,668::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:58,668::__init__::232::IOProcess::(_processLogs) (66) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:23:58,668::__init__::232::IOProcess::(_processLogs) (66) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:58,669::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:58,670::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:58,670::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:58,670::__init__::232::IOProcess::(_processLogs) (67) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:58,670::__init__::232::IOProcess::(_processLogs) (67) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:58,670::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:58,670::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:58,671::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:58,671::__init__::232::IOProcess::(_processLogs) (68) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:58,671::__init__::232::IOProcess::(_processLogs) (68) Queuing response >Thread-16::DEBUG::2014-10-15 07:23:58,671::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:23:58,671::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:23:58,671::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:23:58,672::__init__::232::IOProcess::(_processLogs) (69) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:23:58,672::__init__::232::IOProcess::(_processLogs) (69) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:00,923::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:00,923::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-82::DEBUG::2014-10-15 07:24:00,924::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:03,938::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:03,939::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-83::DEBUG::2014-10-15 07:24:03,939::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:06,953::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:06,954::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-84::DEBUG::2014-10-15 07:24:06,955::task::595::Storage.TaskManager.Task::(_updateState) Task=`b40cf68b-618d-457c-a768-2d560a08ec91`::moving from state init -> state preparing >Thread-84::INFO::2014-10-15 07:24:06,955::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-84::INFO::2014-10-15 07:24:06,956::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00048748', 'lastCheck': '8.3', 'valid': True}} >Thread-84::DEBUG::2014-10-15 07:24:06,956::task::1191::Storage.TaskManager.Task::(prepare) Task=`b40cf68b-618d-457c-a768-2d560a08ec91`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00048748', 'lastCheck': '8.3', 'valid': True}} >Thread-84::DEBUG::2014-10-15 07:24:06,956::task::595::Storage.TaskManager.Task::(_updateState) Task=`b40cf68b-618d-457c-a768-2d560a08ec91`::moving from state preparing -> state finished >Thread-84::DEBUG::2014-10-15 07:24:06,956::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-84::DEBUG::2014-10-15 07:24:06,956::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-84::DEBUG::2014-10-15 07:24:06,956::task::993::Storage.TaskManager.Task::(_decref) Task=`b40cf68b-618d-457c-a768-2d560a08ec91`::ref 0 aborting False >Thread-84::ERROR::2014-10-15 07:24:06,957::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-84::DEBUG::2014-10-15 07:24:06,958::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:06,970::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:06,970::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-85::DEBUG::2014-10-15 07:24:06,971::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:24:08,673::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:08,673::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:08,673::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:08,673::__init__::232::IOProcess::(_processLogs) (70) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:24:08,673::__init__::232::IOProcess::(_processLogs) (70) Queuing response >Thread-29::DEBUG::2014-10-15 07:24:08,674::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:24:08,681::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000387066 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:24:08,681::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:08,681::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:08,682::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:08,682::__init__::232::IOProcess::(_processLogs) (71) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:24:08,682::__init__::232::IOProcess::(_processLogs) (71) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:08,683::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:08,683::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:08,683::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:08,683::__init__::232::IOProcess::(_processLogs) (72) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:08,683::__init__::232::IOProcess::(_processLogs) (72) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:08,683::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:08,684::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:08,684::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:08,684::__init__::232::IOProcess::(_processLogs) (73) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:08,684::__init__::232::IOProcess::(_processLogs) (73) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:08,684::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:08,685::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:08,685::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:08,685::__init__::232::IOProcess::(_processLogs) (74) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:08,685::__init__::232::IOProcess::(_processLogs) (74) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:10,075::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:10,076::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-86::DEBUG::2014-10-15 07:24:10,077::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:13,088::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:13,089::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-87::DEBUG::2014-10-15 07:24:13,090::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:16,101::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:16,102::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-88::DEBUG::2014-10-15 07:24:16,105::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:24:18,686::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:18,686::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:18,686::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:18,686::__init__::232::IOProcess::(_processLogs) (75) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:24:18,686::__init__::232::IOProcess::(_processLogs) (75) Queuing response >Thread-29::DEBUG::2014-10-15 07:24:18,687::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:24:18,694::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000355469 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:24:18,694::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:18,695::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:18,695::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:18,695::__init__::232::IOProcess::(_processLogs) (76) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:24:18,695::__init__::232::IOProcess::(_processLogs) (76) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:18,696::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:18,696::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:18,696::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:18,696::__init__::232::IOProcess::(_processLogs) (77) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:18,696::__init__::232::IOProcess::(_processLogs) (77) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:18,697::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:18,697::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:18,697::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:18,697::__init__::232::IOProcess::(_processLogs) (78) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:18,697::__init__::232::IOProcess::(_processLogs) (78) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:18,697::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:18,698::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:18,698::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:18,698::__init__::232::IOProcess::(_processLogs) (79) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:18,698::__init__::232::IOProcess::(_processLogs) (79) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:19,114::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:19,115::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-89::DEBUG::2014-10-15 07:24:19,116::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:22,125::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:22,125::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-90::DEBUG::2014-10-15 07:24:22,126::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6acc8af-57a8-41bc-8586-a03b784d46f2`::moving from state init -> state preparing >Thread-90::INFO::2014-10-15 07:24:22,127::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-90::INFO::2014-10-15 07:24:22,127::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000355469', 'lastCheck': '3.4', 'valid': True}} >Thread-90::DEBUG::2014-10-15 07:24:22,127::task::1191::Storage.TaskManager.Task::(prepare) Task=`e6acc8af-57a8-41bc-8586-a03b784d46f2`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000355469', 'lastCheck': '3.4', 'valid': True}} >Thread-90::DEBUG::2014-10-15 07:24:22,127::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6acc8af-57a8-41bc-8586-a03b784d46f2`::moving from state preparing -> state finished >Thread-90::DEBUG::2014-10-15 07:24:22,127::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-90::DEBUG::2014-10-15 07:24:22,127::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-90::DEBUG::2014-10-15 07:24:22,127::task::993::Storage.TaskManager.Task::(_decref) Task=`e6acc8af-57a8-41bc-8586-a03b784d46f2`::ref 0 aborting False >Thread-90::ERROR::2014-10-15 07:24:22,128::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-90::DEBUG::2014-10-15 07:24:22,129::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:22,141::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:22,141::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-91::DEBUG::2014-10-15 07:24:22,142::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:25,208::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:25,208::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-92::DEBUG::2014-10-15 07:24:25,210::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:28,220::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:28,221::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-93::DEBUG::2014-10-15 07:24:28,222::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:24:28,699::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:28,699::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:28,699::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:28,699::__init__::232::IOProcess::(_processLogs) (80) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:24:28,699::__init__::232::IOProcess::(_processLogs) (80) Queuing response >Thread-29::DEBUG::2014-10-15 07:24:28,700::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:24:28,706::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000400344 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:24:28,707::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:28,707::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:28,707::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:28,707::__init__::232::IOProcess::(_processLogs) (81) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:24:28,708::__init__::232::IOProcess::(_processLogs) (81) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:28,708::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:28,708::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:28,709::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:28,709::__init__::232::IOProcess::(_processLogs) (82) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:28,709::__init__::232::IOProcess::(_processLogs) (82) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:28,709::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:28,709::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:28,710::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:28,710::__init__::232::IOProcess::(_processLogs) (83) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:28,710::__init__::232::IOProcess::(_processLogs) (83) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:28,710::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:28,710::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:28,710::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:28,711::__init__::232::IOProcess::(_processLogs) (84) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:28,711::__init__::232::IOProcess::(_processLogs) (84) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:31,232::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:31,232::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-94::DEBUG::2014-10-15 07:24:31,233::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:34,245::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:34,246::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-95::DEBUG::2014-10-15 07:24:34,247::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:37,257::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:37,258::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-96::DEBUG::2014-10-15 07:24:37,259::task::595::Storage.TaskManager.Task::(_updateState) Task=`51ee0d5a-2895-4178-ac50-60de85582dee`::moving from state init -> state preparing >Thread-96::INFO::2014-10-15 07:24:37,259::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-96::INFO::2014-10-15 07:24:37,259::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000400344', 'lastCheck': '8.5', 'valid': True}} >Thread-96::DEBUG::2014-10-15 07:24:37,259::task::1191::Storage.TaskManager.Task::(prepare) Task=`51ee0d5a-2895-4178-ac50-60de85582dee`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000400344', 'lastCheck': '8.5', 'valid': True}} >Thread-96::DEBUG::2014-10-15 07:24:37,259::task::595::Storage.TaskManager.Task::(_updateState) Task=`51ee0d5a-2895-4178-ac50-60de85582dee`::moving from state preparing -> state finished >Thread-96::DEBUG::2014-10-15 07:24:37,259::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-96::DEBUG::2014-10-15 07:24:37,260::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-96::DEBUG::2014-10-15 07:24:37,260::task::993::Storage.TaskManager.Task::(_decref) Task=`51ee0d5a-2895-4178-ac50-60de85582dee`::ref 0 aborting False >Thread-96::ERROR::2014-10-15 07:24:37,260::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-96::DEBUG::2014-10-15 07:24:37,261::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:37,273::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:37,273::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-97::DEBUG::2014-10-15 07:24:37,274::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:24:38,712::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:38,712::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:38,712::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:38,712::__init__::232::IOProcess::(_processLogs) (85) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:24:38,713::__init__::232::IOProcess::(_processLogs) (85) Queuing response >Thread-29::DEBUG::2014-10-15 07:24:38,713::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:24:38,720::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000478178 s, 960 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:24:38,721::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:38,721::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:38,721::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:38,721::__init__::232::IOProcess::(_processLogs) (86) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:24:38,722::__init__::232::IOProcess::(_processLogs) (86) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:38,722::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:38,722::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:38,723::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:38,723::__init__::232::IOProcess::(_processLogs) (87) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:38,723::__init__::232::IOProcess::(_processLogs) (87) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:38,723::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:38,723::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:38,724::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:38,724::__init__::232::IOProcess::(_processLogs) (88) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:38,724::__init__::232::IOProcess::(_processLogs) (88) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:38,724::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:38,724::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:38,724::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:38,725::__init__::232::IOProcess::(_processLogs) (89) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:38,725::__init__::232::IOProcess::(_processLogs) (89) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:40,339::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:40,340::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-98::DEBUG::2014-10-15 07:24:40,341::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:43,348::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:43,349::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-99::DEBUG::2014-10-15 07:24:43,350::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:46,361::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:46,361::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-100::DEBUG::2014-10-15 07:24:46,362::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:24:48,726::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:48,726::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:48,726::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:48,726::__init__::232::IOProcess::(_processLogs) (90) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:24:48,726::__init__::232::IOProcess::(_processLogs) (90) Queuing response >Thread-29::DEBUG::2014-10-15 07:24:48,727::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:24:48,734::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000388419 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:24:48,734::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:48,735::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:48,735::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:48,735::__init__::232::IOProcess::(_processLogs) (91) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:24:48,735::__init__::232::IOProcess::(_processLogs) (91) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:48,736::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:48,736::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:48,736::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:48,736::__init__::232::IOProcess::(_processLogs) (92) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:48,736::__init__::232::IOProcess::(_processLogs) (92) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:48,736::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:48,737::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:48,737::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:48,737::__init__::232::IOProcess::(_processLogs) (93) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:48,737::__init__::232::IOProcess::(_processLogs) (93) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:48,737::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:48,738::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:48,738::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:48,738::__init__::232::IOProcess::(_processLogs) (94) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:48,738::__init__::232::IOProcess::(_processLogs) (94) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:49,373::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:49,374::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-101::DEBUG::2014-10-15 07:24:49,375::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:52,386::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:52,386::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-102::DEBUG::2014-10-15 07:24:52,387::task::595::Storage.TaskManager.Task::(_updateState) Task=`cbc06522-7038-4529-842a-2c7958ccecf5`::moving from state init -> state preparing >Thread-102::INFO::2014-10-15 07:24:52,387::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-102::INFO::2014-10-15 07:24:52,388::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000388419', 'lastCheck': '3.6', 'valid': True}} >Thread-102::DEBUG::2014-10-15 07:24:52,388::task::1191::Storage.TaskManager.Task::(prepare) Task=`cbc06522-7038-4529-842a-2c7958ccecf5`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000388419', 'lastCheck': '3.6', 'valid': True}} >Thread-102::DEBUG::2014-10-15 07:24:52,388::task::595::Storage.TaskManager.Task::(_updateState) Task=`cbc06522-7038-4529-842a-2c7958ccecf5`::moving from state preparing -> state finished >Thread-102::DEBUG::2014-10-15 07:24:52,388::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-102::DEBUG::2014-10-15 07:24:52,388::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-102::DEBUG::2014-10-15 07:24:52,388::task::993::Storage.TaskManager.Task::(_decref) Task=`cbc06522-7038-4529-842a-2c7958ccecf5`::ref 0 aborting False >Thread-102::ERROR::2014-10-15 07:24:52,389::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-102::DEBUG::2014-10-15 07:24:52,390::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:52,402::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:52,402::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-103::DEBUG::2014-10-15 07:24:52,403::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:55,463::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:55,463::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-104::DEBUG::2014-10-15 07:24:55,464::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:24:58,474::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:24:58,475::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-105::DEBUG::2014-10-15 07:24:58,476::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:24:58,739::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:58,739::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:58,739::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:58,739::__init__::232::IOProcess::(_processLogs) (95) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:24:58,739::__init__::232::IOProcess::(_processLogs) (95) Queuing response >Thread-29::DEBUG::2014-10-15 07:24:58,740::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:24:58,746::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000382506 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:24:58,747::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:58,747::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:58,747::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:58,748::__init__::232::IOProcess::(_processLogs) (96) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:24:58,748::__init__::232::IOProcess::(_processLogs) (96) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:58,748::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:58,749::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:58,749::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:58,749::__init__::232::IOProcess::(_processLogs) (97) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:58,749::__init__::232::IOProcess::(_processLogs) (97) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:58,749::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:58,750::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:58,750::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:58,750::__init__::232::IOProcess::(_processLogs) (98) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:58,750::__init__::232::IOProcess::(_processLogs) (98) Queuing response >Thread-16::DEBUG::2014-10-15 07:24:58,750::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:24:58,750::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:24:58,750::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:24:58,751::__init__::232::IOProcess::(_processLogs) (99) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:24:58,751::__init__::232::IOProcess::(_processLogs) (99) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:01,483::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:01,484::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-106::DEBUG::2014-10-15 07:25:01,485::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:04,493::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:04,494::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-107::DEBUG::2014-10-15 07:25:04,495::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:07,503::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:07,504::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-108::DEBUG::2014-10-15 07:25:07,505::task::595::Storage.TaskManager.Task::(_updateState) Task=`cfea59f6-aeb1-4f4e-9f30-a4974bada7e1`::moving from state init -> state preparing >Thread-108::INFO::2014-10-15 07:25:07,505::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-10-15 07:25:07,505::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000382506', 'lastCheck': '8.8', 'valid': True}} >Thread-108::DEBUG::2014-10-15 07:25:07,506::task::1191::Storage.TaskManager.Task::(prepare) Task=`cfea59f6-aeb1-4f4e-9f30-a4974bada7e1`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000382506', 'lastCheck': '8.8', 'valid': True}} >Thread-108::DEBUG::2014-10-15 07:25:07,506::task::595::Storage.TaskManager.Task::(_updateState) Task=`cfea59f6-aeb1-4f4e-9f30-a4974bada7e1`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-10-15 07:25:07,506::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-10-15 07:25:07,506::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-10-15 07:25:07,506::task::993::Storage.TaskManager.Task::(_decref) Task=`cfea59f6-aeb1-4f4e-9f30-a4974bada7e1`::ref 0 aborting False >Thread-108::ERROR::2014-10-15 07:25:07,507::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-108::DEBUG::2014-10-15 07:25:07,508::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:07,526::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:07,526::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-109::DEBUG::2014-10-15 07:25:07,527::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:25:08,752::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:08,752::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:08,752::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:08,752::__init__::232::IOProcess::(_processLogs) (100) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:25:08,753::__init__::232::IOProcess::(_processLogs) (100) Queuing response >Thread-29::DEBUG::2014-10-15 07:25:08,753::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:25:08,760::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000361251 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:25:08,760::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:08,761::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:08,761::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:08,761::__init__::232::IOProcess::(_processLogs) (101) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:25:08,761::__init__::232::IOProcess::(_processLogs) (101) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:08,762::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:08,762::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:08,762::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:08,763::__init__::232::IOProcess::(_processLogs) (102) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:08,763::__init__::232::IOProcess::(_processLogs) (102) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:08,763::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:08,763::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:08,763::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:08,763::__init__::232::IOProcess::(_processLogs) (103) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:08,764::__init__::232::IOProcess::(_processLogs) (103) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:08,764::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:08,764::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:08,764::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:08,764::__init__::232::IOProcess::(_processLogs) (104) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:08,764::__init__::232::IOProcess::(_processLogs) (104) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:10,594::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:10,594::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-110::DEBUG::2014-10-15 07:25:10,595::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:13,604::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:13,605::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-111::DEBUG::2014-10-15 07:25:13,606::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:16,614::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:16,614::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-112::DEBUG::2014-10-15 07:25:16,615::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:25:18,765::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:18,765::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:18,766::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:18,766::__init__::232::IOProcess::(_processLogs) (105) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:25:18,766::__init__::232::IOProcess::(_processLogs) (105) Queuing response >Thread-29::DEBUG::2014-10-15 07:25:18,766::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:25:18,773::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000405225 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:25:18,774::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:18,774::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:18,774::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:18,774::__init__::232::IOProcess::(_processLogs) (106) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:25:18,774::__init__::232::IOProcess::(_processLogs) (106) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:18,775::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:18,775::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:18,776::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:18,776::__init__::232::IOProcess::(_processLogs) (107) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:18,776::__init__::232::IOProcess::(_processLogs) (107) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:18,776::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:18,776::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:18,777::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:18,777::__init__::232::IOProcess::(_processLogs) (108) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:18,777::__init__::232::IOProcess::(_processLogs) (108) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:18,777::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:18,777::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:18,777::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:18,777::__init__::232::IOProcess::(_processLogs) (109) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:18,777::__init__::232::IOProcess::(_processLogs) (109) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:19,623::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:19,624::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-113::DEBUG::2014-10-15 07:25:19,625::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:22,634::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:22,635::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-114::DEBUG::2014-10-15 07:25:22,636::task::595::Storage.TaskManager.Task::(_updateState) Task=`32e23a90-b744-4508-bbb8-b18c85a9a490`::moving from state init -> state preparing >Thread-114::INFO::2014-10-15 07:25:22,636::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-114::INFO::2014-10-15 07:25:22,636::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000405225', 'lastCheck': '3.9', 'valid': True}} >Thread-114::DEBUG::2014-10-15 07:25:22,636::task::1191::Storage.TaskManager.Task::(prepare) Task=`32e23a90-b744-4508-bbb8-b18c85a9a490`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000405225', 'lastCheck': '3.9', 'valid': True}} >Thread-114::DEBUG::2014-10-15 07:25:22,636::task::595::Storage.TaskManager.Task::(_updateState) Task=`32e23a90-b744-4508-bbb8-b18c85a9a490`::moving from state preparing -> state finished >Thread-114::DEBUG::2014-10-15 07:25:22,637::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-114::DEBUG::2014-10-15 07:25:22,637::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-114::DEBUG::2014-10-15 07:25:22,637::task::993::Storage.TaskManager.Task::(_decref) Task=`32e23a90-b744-4508-bbb8-b18c85a9a490`::ref 0 aborting False >Thread-114::ERROR::2014-10-15 07:25:22,638::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-114::DEBUG::2014-10-15 07:25:22,638::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:22,647::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:22,648::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-115::DEBUG::2014-10-15 07:25:22,649::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:25,715::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:25,716::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-116::DEBUG::2014-10-15 07:25:25,717::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:28,735::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:28,736::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-117::DEBUG::2014-10-15 07:25:28,739::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:25:28,778::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:28,778::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:28,778::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:28,779::__init__::232::IOProcess::(_processLogs) (110) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:25:28,779::__init__::232::IOProcess::(_processLogs) (110) Queuing response >Thread-29::DEBUG::2014-10-15 07:25:28,779::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:25:28,786::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000376836 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:25:28,787::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:28,787::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:28,787::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:28,787::__init__::232::IOProcess::(_processLogs) (111) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:25:28,787::__init__::232::IOProcess::(_processLogs) (111) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:28,788::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:28,788::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:28,789::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:28,789::__init__::232::IOProcess::(_processLogs) (112) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:28,789::__init__::232::IOProcess::(_processLogs) (112) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:28,789::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:28,789::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:28,790::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:28,790::__init__::232::IOProcess::(_processLogs) (113) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:28,790::__init__::232::IOProcess::(_processLogs) (113) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:28,790::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:28,790::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:28,790::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:28,791::__init__::232::IOProcess::(_processLogs) (114) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:28,791::__init__::232::IOProcess::(_processLogs) (114) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:31,747::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:31,748::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-118::DEBUG::2014-10-15 07:25:31,749::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:34,759::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:34,760::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-119::DEBUG::2014-10-15 07:25:34,761::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:37,770::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:37,772::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-120::DEBUG::2014-10-15 07:25:37,773::task::595::Storage.TaskManager.Task::(_updateState) Task=`fe57fcac-bb0e-4938-ac0e-726be9deab9c`::moving from state init -> state preparing >Thread-120::INFO::2014-10-15 07:25:37,773::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-120::INFO::2014-10-15 07:25:37,773::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000376836', 'lastCheck': '9.0', 'valid': True}} >Thread-120::DEBUG::2014-10-15 07:25:37,773::task::1191::Storage.TaskManager.Task::(prepare) Task=`fe57fcac-bb0e-4938-ac0e-726be9deab9c`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000376836', 'lastCheck': '9.0', 'valid': True}} >Thread-120::DEBUG::2014-10-15 07:25:37,774::task::595::Storage.TaskManager.Task::(_updateState) Task=`fe57fcac-bb0e-4938-ac0e-726be9deab9c`::moving from state preparing -> state finished >Thread-120::DEBUG::2014-10-15 07:25:37,774::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-120::DEBUG::2014-10-15 07:25:37,774::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-120::DEBUG::2014-10-15 07:25:37,774::task::993::Storage.TaskManager.Task::(_decref) Task=`fe57fcac-bb0e-4938-ac0e-726be9deab9c`::ref 0 aborting False >Thread-120::ERROR::2014-10-15 07:25:37,775::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-120::DEBUG::2014-10-15 07:25:37,775::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:37,786::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:37,786::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-121::DEBUG::2014-10-15 07:25:37,787::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:25:38,791::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:38,792::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:38,792::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:38,792::__init__::232::IOProcess::(_processLogs) (115) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:25:38,793::__init__::232::IOProcess::(_processLogs) (115) Queuing response >Thread-29::DEBUG::2014-10-15 07:25:38,793::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:25:38,801::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000443683 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:25:38,801::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:38,802::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:38,802::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:38,802::__init__::232::IOProcess::(_processLogs) (116) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:25:38,802::__init__::232::IOProcess::(_processLogs) (116) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:38,803::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:38,803::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:38,803::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:38,803::__init__::232::IOProcess::(_processLogs) (117) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:38,804::__init__::232::IOProcess::(_processLogs) (117) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:38,804::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:38,804::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:38,804::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:38,804::__init__::232::IOProcess::(_processLogs) (118) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:38,804::__init__::232::IOProcess::(_processLogs) (118) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:38,805::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:38,805::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:38,805::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:38,805::__init__::232::IOProcess::(_processLogs) (119) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:38,805::__init__::232::IOProcess::(_processLogs) (119) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:40,852::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:40,853::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-122::DEBUG::2014-10-15 07:25:40,854::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:43,864::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:43,865::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-123::DEBUG::2014-10-15 07:25:43,866::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:46,876::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:46,876::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-124::DEBUG::2014-10-15 07:25:46,877::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:25:48,806::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:48,807::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:48,807::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:48,807::__init__::232::IOProcess::(_processLogs) (120) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:25:48,807::__init__::232::IOProcess::(_processLogs) (120) Queuing response >Thread-29::DEBUG::2014-10-15 07:25:48,808::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:25:48,815::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000447062 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:25:48,815::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:48,815::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:48,815::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:48,816::__init__::232::IOProcess::(_processLogs) (121) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:25:48,816::__init__::232::IOProcess::(_processLogs) (121) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:48,816::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:48,817::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:48,817::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:48,817::__init__::232::IOProcess::(_processLogs) (122) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:48,817::__init__::232::IOProcess::(_processLogs) (122) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:48,817::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:48,817::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:48,818::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:48,818::__init__::232::IOProcess::(_processLogs) (123) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:48,818::__init__::232::IOProcess::(_processLogs) (123) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:48,818::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:48,818::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:48,819::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:48,819::__init__::232::IOProcess::(_processLogs) (124) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:48,819::__init__::232::IOProcess::(_processLogs) (124) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:49,887::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:49,888::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-125::DEBUG::2014-10-15 07:25:49,889::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:52,899::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:52,900::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-126::DEBUG::2014-10-15 07:25:52,901::task::595::Storage.TaskManager.Task::(_updateState) Task=`a37ead04-7678-4f7f-b2da-6243982baf7c`::moving from state init -> state preparing >Thread-126::INFO::2014-10-15 07:25:52,901::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-10-15 07:25:52,901::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000447062', 'lastCheck': '4.1', 'valid': True}} >Thread-126::DEBUG::2014-10-15 07:25:52,901::task::1191::Storage.TaskManager.Task::(prepare) Task=`a37ead04-7678-4f7f-b2da-6243982baf7c`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000447062', 'lastCheck': '4.1', 'valid': True}} >Thread-126::DEBUG::2014-10-15 07:25:52,902::task::595::Storage.TaskManager.Task::(_updateState) Task=`a37ead04-7678-4f7f-b2da-6243982baf7c`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-10-15 07:25:52,902::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-10-15 07:25:52,902::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-10-15 07:25:52,902::task::993::Storage.TaskManager.Task::(_decref) Task=`a37ead04-7678-4f7f-b2da-6243982baf7c`::ref 0 aborting False >Thread-126::ERROR::2014-10-15 07:25:52,903::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-126::DEBUG::2014-10-15 07:25:52,903::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:52,915::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:52,915::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-127::DEBUG::2014-10-15 07:25:52,916::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:55,984::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:55,985::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-128::DEBUG::2014-10-15 07:25:55,985::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:25:58,820::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:58,820::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:58,820::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:58,820::__init__::232::IOProcess::(_processLogs) (125) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:25:58,820::__init__::232::IOProcess::(_processLogs) (125) Queuing response >Thread-29::DEBUG::2014-10-15 07:25:58,821::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:25:58,828::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000473585 s, 969 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:25:58,828::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:58,829::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:58,829::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:58,829::__init__::232::IOProcess::(_processLogs) (126) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:25:58,829::__init__::232::IOProcess::(_processLogs) (126) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:58,830::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:58,830::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:58,830::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:58,830::__init__::232::IOProcess::(_processLogs) (127) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:58,830::__init__::232::IOProcess::(_processLogs) (127) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:58,831::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:58,831::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:58,831::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:58,831::__init__::232::IOProcess::(_processLogs) (128) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:58,831::__init__::232::IOProcess::(_processLogs) (128) Queuing response >Thread-16::DEBUG::2014-10-15 07:25:58,831::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:25:58,832::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:25:58,832::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:25:58,832::__init__::232::IOProcess::(_processLogs) (129) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:25:58,832::__init__::232::IOProcess::(_processLogs) (129) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:25:58,996::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:25:58,997::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-129::DEBUG::2014-10-15 07:25:58,998::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:02,008::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:02,009::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-130::DEBUG::2014-10-15 07:26:02,010::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:05,020::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:05,021::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-131::DEBUG::2014-10-15 07:26:05,022::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:08,032::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:08,033::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-132::DEBUG::2014-10-15 07:26:08,034::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8e16ce8-8fee-411b-adf0-6fb1fa03a0bf`::moving from state init -> state preparing >Thread-132::INFO::2014-10-15 07:26:08,034::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-132::INFO::2014-10-15 07:26:08,034::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000473585', 'lastCheck': '9.2', 'valid': True}} >Thread-132::DEBUG::2014-10-15 07:26:08,034::task::1191::Storage.TaskManager.Task::(prepare) Task=`b8e16ce8-8fee-411b-adf0-6fb1fa03a0bf`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000473585', 'lastCheck': '9.2', 'valid': True}} >Thread-132::DEBUG::2014-10-15 07:26:08,034::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8e16ce8-8fee-411b-adf0-6fb1fa03a0bf`::moving from state preparing -> state finished >Thread-132::DEBUG::2014-10-15 07:26:08,034::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2014-10-15 07:26:08,034::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-132::DEBUG::2014-10-15 07:26:08,035::task::993::Storage.TaskManager.Task::(_decref) Task=`b8e16ce8-8fee-411b-adf0-6fb1fa03a0bf`::ref 0 aborting False >Thread-132::ERROR::2014-10-15 07:26:08,035::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-132::DEBUG::2014-10-15 07:26:08,036::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:08,071::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:08,072::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-133::DEBUG::2014-10-15 07:26:08,073::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:26:08,833::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:08,833::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:08,833::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:08,833::__init__::232::IOProcess::(_processLogs) (130) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:26:08,833::__init__::232::IOProcess::(_processLogs) (130) Queuing response >Thread-29::DEBUG::2014-10-15 07:26:08,834::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:26:08,841::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00044992 s, 1.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:26:08,841::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:08,841::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:08,842::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:08,842::__init__::232::IOProcess::(_processLogs) (131) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:26:08,842::__init__::232::IOProcess::(_processLogs) (131) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:08,843::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:08,843::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:08,843::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:08,843::__init__::232::IOProcess::(_processLogs) (132) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:08,843::__init__::232::IOProcess::(_processLogs) (132) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:08,843::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:08,844::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:08,844::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:08,844::__init__::232::IOProcess::(_processLogs) (133) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:08,844::__init__::232::IOProcess::(_processLogs) (133) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:08,844::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:08,845::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:08,845::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:08,845::__init__::232::IOProcess::(_processLogs) (134) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:08,845::__init__::232::IOProcess::(_processLogs) (134) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:11,130::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:11,131::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-134::DEBUG::2014-10-15 07:26:11,133::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:14,143::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:14,144::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-135::DEBUG::2014-10-15 07:26:14,145::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:17,154::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:17,155::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-136::DEBUG::2014-10-15 07:26:17,156::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:26:18,846::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:18,846::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:18,846::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:18,846::__init__::232::IOProcess::(_processLogs) (135) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:26:18,846::__init__::232::IOProcess::(_processLogs) (135) Queuing response >Thread-29::DEBUG::2014-10-15 07:26:18,847::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:26:18,854::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00036874 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:26:18,854::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:18,855::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:18,855::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:18,855::__init__::232::IOProcess::(_processLogs) (136) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:26:18,855::__init__::232::IOProcess::(_processLogs) (136) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:18,856::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:18,856::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:18,856::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:18,856::__init__::232::IOProcess::(_processLogs) (137) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:18,856::__init__::232::IOProcess::(_processLogs) (137) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:18,857::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:18,857::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:18,857::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:18,857::__init__::232::IOProcess::(_processLogs) (138) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:18,857::__init__::232::IOProcess::(_processLogs) (138) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:18,857::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:18,858::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:18,858::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:18,858::__init__::232::IOProcess::(_processLogs) (139) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:18,858::__init__::232::IOProcess::(_processLogs) (139) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:20,166::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:20,167::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-137::DEBUG::2014-10-15 07:26:20,168::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:23,174::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:23,175::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-138::DEBUG::2014-10-15 07:26:23,176::task::595::Storage.TaskManager.Task::(_updateState) Task=`bae16e66-cf38-4117-adb0-7b022590d619`::moving from state init -> state preparing >Thread-138::INFO::2014-10-15 07:26:23,176::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-138::INFO::2014-10-15 07:26:23,176::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00036874', 'lastCheck': '4.3', 'valid': True}} >Thread-138::DEBUG::2014-10-15 07:26:23,176::task::1191::Storage.TaskManager.Task::(prepare) Task=`bae16e66-cf38-4117-adb0-7b022590d619`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00036874', 'lastCheck': '4.3', 'valid': True}} >Thread-138::DEBUG::2014-10-15 07:26:23,176::task::595::Storage.TaskManager.Task::(_updateState) Task=`bae16e66-cf38-4117-adb0-7b022590d619`::moving from state preparing -> state finished >Thread-138::DEBUG::2014-10-15 07:26:23,176::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-138::DEBUG::2014-10-15 07:26:23,177::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-138::DEBUG::2014-10-15 07:26:23,177::task::993::Storage.TaskManager.Task::(_decref) Task=`bae16e66-cf38-4117-adb0-7b022590d619`::ref 0 aborting False >Thread-138::ERROR::2014-10-15 07:26:23,177::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-138::DEBUG::2014-10-15 07:26:23,178::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:23,185::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:23,186::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-139::DEBUG::2014-10-15 07:26:23,187::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:26,251::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:26,251::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-140::DEBUG::2014-10-15 07:26:26,252::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:26:28,859::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:28,859::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:28,859::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:28,859::__init__::232::IOProcess::(_processLogs) (140) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:26:28,859::__init__::232::IOProcess::(_processLogs) (140) Queuing response >Thread-29::DEBUG::2014-10-15 07:26:28,860::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:26:28,868::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00113241 s, 405 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:26:28,869::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:28,869::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:28,869::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:28,869::__init__::232::IOProcess::(_processLogs) (141) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:26:28,871::__init__::232::IOProcess::(_processLogs) (141) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:28,871::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:28,871::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:28,872::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:28,872::__init__::232::IOProcess::(_processLogs) (142) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:28,872::__init__::232::IOProcess::(_processLogs) (142) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:28,872::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:28,872::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:28,873::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:28,873::__init__::232::IOProcess::(_processLogs) (143) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:28,873::__init__::232::IOProcess::(_processLogs) (143) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:28,873::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:28,873::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:28,873::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:28,874::__init__::232::IOProcess::(_processLogs) (144) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:28,874::__init__::232::IOProcess::(_processLogs) (144) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:29,262::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:29,263::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-141::DEBUG::2014-10-15 07:26:29,264::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:32,275::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:32,275::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-142::DEBUG::2014-10-15 07:26:32,276::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:35,286::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:35,287::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-143::DEBUG::2014-10-15 07:26:35,288::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:38,297::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:38,298::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-144::DEBUG::2014-10-15 07:26:38,299::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7723a69-da45-493a-9956-b8c2d5ebed73`::moving from state init -> state preparing >Thread-144::INFO::2014-10-15 07:26:38,300::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-144::INFO::2014-10-15 07:26:38,300::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00113241', 'lastCheck': '9.4', 'valid': True}} >Thread-144::DEBUG::2014-10-15 07:26:38,300::task::1191::Storage.TaskManager.Task::(prepare) Task=`e7723a69-da45-493a-9956-b8c2d5ebed73`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00113241', 'lastCheck': '9.4', 'valid': True}} >Thread-144::DEBUG::2014-10-15 07:26:38,300::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7723a69-da45-493a-9956-b8c2d5ebed73`::moving from state preparing -> state finished >Thread-144::DEBUG::2014-10-15 07:26:38,300::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-144::DEBUG::2014-10-15 07:26:38,300::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-144::DEBUG::2014-10-15 07:26:38,300::task::993::Storage.TaskManager.Task::(_decref) Task=`e7723a69-da45-493a-9956-b8c2d5ebed73`::ref 0 aborting False >Thread-144::ERROR::2014-10-15 07:26:38,301::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-144::DEBUG::2014-10-15 07:26:38,302::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:38,322::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:38,322::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-145::DEBUG::2014-10-15 07:26:38,323::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:26:38,874::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:38,875::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:38,875::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:38,875::__init__::232::IOProcess::(_processLogs) (145) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:26:38,875::__init__::232::IOProcess::(_processLogs) (145) Queuing response >Thread-29::DEBUG::2014-10-15 07:26:38,876::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:26:38,883::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000474201 s, 968 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:26:38,884::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:38,884::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:38,884::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:38,884::__init__::232::IOProcess::(_processLogs) (146) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:26:38,885::__init__::232::IOProcess::(_processLogs) (146) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:38,885::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:38,885::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:38,886::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:38,886::__init__::232::IOProcess::(_processLogs) (147) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:38,886::__init__::232::IOProcess::(_processLogs) (147) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:38,886::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:38,886::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:38,886::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:38,887::__init__::232::IOProcess::(_processLogs) (148) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:38,887::__init__::232::IOProcess::(_processLogs) (148) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:38,887::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:38,887::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:38,887::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:38,887::__init__::232::IOProcess::(_processLogs) (149) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:38,888::__init__::232::IOProcess::(_processLogs) (149) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:41,381::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:41,381::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-146::DEBUG::2014-10-15 07:26:41,382::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:44,393::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:44,394::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-147::DEBUG::2014-10-15 07:26:44,395::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:47,406::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:47,407::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-148::DEBUG::2014-10-15 07:26:47,408::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:26:48,889::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:48,889::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:48,889::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:48,889::__init__::232::IOProcess::(_processLogs) (150) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:26:48,889::__init__::232::IOProcess::(_processLogs) (150) Queuing response >Thread-29::DEBUG::2014-10-15 07:26:48,890::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:26:48,896::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000435104 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:26:48,897::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:48,897::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:48,897::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:48,897::__init__::232::IOProcess::(_processLogs) (151) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:26:48,898::__init__::232::IOProcess::(_processLogs) (151) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:48,898::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:48,899::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:48,899::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:48,899::__init__::232::IOProcess::(_processLogs) (152) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:48,899::__init__::232::IOProcess::(_processLogs) (152) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:48,899::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:48,900::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:48,900::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:48,900::__init__::232::IOProcess::(_processLogs) (153) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:48,900::__init__::232::IOProcess::(_processLogs) (153) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:48,900::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:48,900::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:48,901::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:48,901::__init__::232::IOProcess::(_processLogs) (154) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:48,901::__init__::232::IOProcess::(_processLogs) (154) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:50,418::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:50,419::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-149::DEBUG::2014-10-15 07:26:50,420::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:53,429::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:53,430::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-150::DEBUG::2014-10-15 07:26:53,431::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ad2cbeb-9db2-48df-b0e8-0a17f93f0960`::moving from state init -> state preparing >Thread-150::INFO::2014-10-15 07:26:53,431::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-150::INFO::2014-10-15 07:26:53,431::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000435104', 'lastCheck': '4.5', 'valid': True}} >Thread-150::DEBUG::2014-10-15 07:26:53,431::task::1191::Storage.TaskManager.Task::(prepare) Task=`3ad2cbeb-9db2-48df-b0e8-0a17f93f0960`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000435104', 'lastCheck': '4.5', 'valid': True}} >Thread-150::DEBUG::2014-10-15 07:26:53,431::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ad2cbeb-9db2-48df-b0e8-0a17f93f0960`::moving from state preparing -> state finished >Thread-150::DEBUG::2014-10-15 07:26:53,432::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-150::DEBUG::2014-10-15 07:26:53,432::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-150::DEBUG::2014-10-15 07:26:53,432::task::993::Storage.TaskManager.Task::(_decref) Task=`3ad2cbeb-9db2-48df-b0e8-0a17f93f0960`::ref 0 aborting False >Thread-150::ERROR::2014-10-15 07:26:53,433::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-150::DEBUG::2014-10-15 07:26:53,433::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:53,442::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:53,442::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-151::DEBUG::2014-10-15 07:26:53,443::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:56,579::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:56,580::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-152::DEBUG::2014-10-15 07:26:56,581::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:26:58,902::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:58,902::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:58,902::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:58,902::__init__::232::IOProcess::(_processLogs) (155) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:26:58,903::__init__::232::IOProcess::(_processLogs) (155) Queuing response >Thread-29::DEBUG::2014-10-15 07:26:58,903::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:26:58,910::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.00040851 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:26:58,910::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:58,910::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:58,911::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:58,911::__init__::232::IOProcess::(_processLogs) (156) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:26:58,911::__init__::232::IOProcess::(_processLogs) (156) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:58,912::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:58,912::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:58,912::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:58,912::__init__::232::IOProcess::(_processLogs) (157) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:58,912::__init__::232::IOProcess::(_processLogs) (157) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:58,913::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:58,913::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:58,913::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:58,913::__init__::232::IOProcess::(_processLogs) (158) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:58,913::__init__::232::IOProcess::(_processLogs) (158) Queuing response >Thread-16::DEBUG::2014-10-15 07:26:58,914::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:26:58,914::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:26:58,914::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:26:58,914::__init__::232::IOProcess::(_processLogs) (159) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:26:58,914::__init__::232::IOProcess::(_processLogs) (159) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:26:59,594::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:26:59,595::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-153::DEBUG::2014-10-15 07:26:59,596::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:02,604::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:02,604::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-154::DEBUG::2014-10-15 07:27:02,605::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:05,614::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:05,615::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-155::DEBUG::2014-10-15 07:27:05,616::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:08,625::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:08,626::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-156::DEBUG::2014-10-15 07:27:08,627::task::595::Storage.TaskManager.Task::(_updateState) Task=`51a20ad9-26fc-4a2f-87ab-574af0c9efea`::moving from state init -> state preparing >Thread-156::INFO::2014-10-15 07:27:08,627::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-156::INFO::2014-10-15 07:27:08,627::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00040851', 'lastCheck': '9.7', 'valid': True}} >Thread-156::DEBUG::2014-10-15 07:27:08,627::task::1191::Storage.TaskManager.Task::(prepare) Task=`51a20ad9-26fc-4a2f-87ab-574af0c9efea`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00040851', 'lastCheck': '9.7', 'valid': True}} >Thread-156::DEBUG::2014-10-15 07:27:08,628::task::595::Storage.TaskManager.Task::(_updateState) Task=`51a20ad9-26fc-4a2f-87ab-574af0c9efea`::moving from state preparing -> state finished >Thread-156::DEBUG::2014-10-15 07:27:08,628::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-156::DEBUG::2014-10-15 07:27:08,628::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-156::DEBUG::2014-10-15 07:27:08,628::task::993::Storage.TaskManager.Task::(_decref) Task=`51a20ad9-26fc-4a2f-87ab-574af0c9efea`::ref 0 aborting False >Thread-156::ERROR::2014-10-15 07:27:08,629::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-156::DEBUG::2014-10-15 07:27:08,629::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:08,640::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:08,640::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-157::DEBUG::2014-10-15 07:27:08,641::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:27:08,915::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:08,915::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:08,915::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:08,916::__init__::232::IOProcess::(_processLogs) (160) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:27:08,916::__init__::232::IOProcess::(_processLogs) (160) Queuing response >Thread-29::DEBUG::2014-10-15 07:27:08,916::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:27:08,923::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000382829 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:27:08,923::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:08,924::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:08,924::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:08,924::__init__::232::IOProcess::(_processLogs) (161) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:27:08,924::__init__::232::IOProcess::(_processLogs) (161) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:08,925::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:08,925::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:08,925::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:08,925::__init__::232::IOProcess::(_processLogs) (162) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:08,926::__init__::232::IOProcess::(_processLogs) (162) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:08,926::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:08,926::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:08,926::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:08,926::__init__::232::IOProcess::(_processLogs) (163) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:08,926::__init__::232::IOProcess::(_processLogs) (163) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:08,927::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:08,927::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:08,927::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:08,927::__init__::232::IOProcess::(_processLogs) (164) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:08,927::__init__::232::IOProcess::(_processLogs) (164) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:11,774::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:11,775::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-158::DEBUG::2014-10-15 07:27:11,776::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:14,785::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:14,786::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-159::DEBUG::2014-10-15 07:27:14,787::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:17,798::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:17,798::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-160::DEBUG::2014-10-15 07:27:17,799::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:27:18,928::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:18,928::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:18,928::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:18,928::__init__::232::IOProcess::(_processLogs) (165) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:27:18,929::__init__::232::IOProcess::(_processLogs) (165) Queuing response >Thread-29::DEBUG::2014-10-15 07:27:18,929::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:27:18,936::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000370714 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:27:18,937::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:18,937::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:18,937::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:18,937::__init__::232::IOProcess::(_processLogs) (166) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:27:18,937::__init__::232::IOProcess::(_processLogs) (166) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:18,938::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:18,938::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:18,938::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:18,938::__init__::232::IOProcess::(_processLogs) (167) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:18,939::__init__::232::IOProcess::(_processLogs) (167) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:18,939::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:18,939::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:18,939::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:18,939::__init__::232::IOProcess::(_processLogs) (168) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:18,939::__init__::232::IOProcess::(_processLogs) (168) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:18,940::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:18,940::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:18,940::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:18,940::__init__::232::IOProcess::(_processLogs) (169) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:18,940::__init__::232::IOProcess::(_processLogs) (169) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:20,810::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:20,811::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-161::DEBUG::2014-10-15 07:27:20,812::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:23,822::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:23,823::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-162::DEBUG::2014-10-15 07:27:23,824::task::595::Storage.TaskManager.Task::(_updateState) Task=`5451dda9-af12-4d76-b39c-a6d3f41204e3`::moving from state init -> state preparing >Thread-162::INFO::2014-10-15 07:27:23,824::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-162::INFO::2014-10-15 07:27:23,825::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000370714', 'lastCheck': '4.9', 'valid': True}} >Thread-162::DEBUG::2014-10-15 07:27:23,825::task::1191::Storage.TaskManager.Task::(prepare) Task=`5451dda9-af12-4d76-b39c-a6d3f41204e3`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000370714', 'lastCheck': '4.9', 'valid': True}} >Thread-162::DEBUG::2014-10-15 07:27:23,825::task::595::Storage.TaskManager.Task::(_updateState) Task=`5451dda9-af12-4d76-b39c-a6d3f41204e3`::moving from state preparing -> state finished >Thread-162::DEBUG::2014-10-15 07:27:23,825::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-162::DEBUG::2014-10-15 07:27:23,825::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-162::DEBUG::2014-10-15 07:27:23,825::task::993::Storage.TaskManager.Task::(_decref) Task=`5451dda9-af12-4d76-b39c-a6d3f41204e3`::ref 0 aborting False >Thread-162::ERROR::2014-10-15 07:27:23,826::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-162::DEBUG::2014-10-15 07:27:23,827::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:23,834::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:23,835::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-163::DEBUG::2014-10-15 07:27:23,835::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:26,916::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:26,917::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-164::DEBUG::2014-10-15 07:27:26,918::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:27:28,941::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:28,941::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:28,941::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:28,941::__init__::232::IOProcess::(_processLogs) (170) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:27:28,942::__init__::232::IOProcess::(_processLogs) (170) Queuing response >Thread-29::DEBUG::2014-10-15 07:27:28,942::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:27:28,949::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000407663 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:27:28,950::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:28,950::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:28,950::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:28,950::__init__::232::IOProcess::(_processLogs) (171) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:27:28,950::__init__::232::IOProcess::(_processLogs) (171) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:28,951::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:28,951::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:28,952::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:28,952::__init__::232::IOProcess::(_processLogs) (172) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:28,952::__init__::232::IOProcess::(_processLogs) (172) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:28,952::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:28,952::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:28,952::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:28,953::__init__::232::IOProcess::(_processLogs) (173) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:28,953::__init__::232::IOProcess::(_processLogs) (173) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:28,953::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:28,953::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:28,953::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:28,953::__init__::232::IOProcess::(_processLogs) (174) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:28,954::__init__::232::IOProcess::(_processLogs) (174) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:29,926::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:29,927::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-165::DEBUG::2014-10-15 07:27:29,927::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:32,938::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:32,938::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-166::DEBUG::2014-10-15 07:27:32,939::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:35,950::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:35,951::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-167::DEBUG::2014-10-15 07:27:35,952::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-29::DEBUG::2014-10-15 07:27:38,954::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 07:27:38,955::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:38,955::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:38,955::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:38,955::__init__::232::IOProcess::(_processLogs) (175) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:27:38,956::__init__::232::IOProcess::(_processLogs) (175) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:38,962::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:38,963::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-169::DEBUG::2014-10-15 07:27:38,967::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a4e5dfb-0aa0-4400-8fdd-b2eb645e4fb0`::moving from state init -> state preparing >Thread-169::INFO::2014-10-15 07:27:38,968::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-169::INFO::2014-10-15 07:27:38,969::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000407663', 'lastCheck': '10.0', 'valid': True}} >Thread-169::DEBUG::2014-10-15 07:27:38,969::task::1191::Storage.TaskManager.Task::(prepare) Task=`7a4e5dfb-0aa0-4400-8fdd-b2eb645e4fb0`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000407663', 'lastCheck': '10.0', 'valid': True}} >Thread-169::DEBUG::2014-10-15 07:27:38,970::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a4e5dfb-0aa0-4400-8fdd-b2eb645e4fb0`::moving from state preparing -> state finished >Thread-169::DEBUG::2014-10-15 07:27:38,970::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-169::DEBUG::2014-10-15 07:27:38,971::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-169::DEBUG::2014-10-15 07:27:38,971::task::993::Storage.TaskManager.Task::(_decref) Task=`7a4e5dfb-0aa0-4400-8fdd-b2eb645e4fb0`::ref 0 aborting False >Thread-29::DEBUG::2014-10-15 07:27:38,972::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-16::DEBUG::2014-10-15 07:27:38,973::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:38,974::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-169::ERROR::2014-10-15 07:27:38,974::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-16::DEBUG::2014-10-15 07:27:38,975::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:38,975::__init__::232::IOProcess::(_processLogs) (176) Got request for method 'touch' >Thread-169::DEBUG::2014-10-15 07:27:38,976::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:38,985::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:38,985::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-170::DEBUG::2014-10-15 07:27:38,986::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:27:39,000::__init__::232::IOProcess::(_processLogs) (176) Queuing response >Thread-29::DEBUG::2014-10-15 07:27:39,001::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-16::DEBUG::2014-10-15 07:27:39,001::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:39,001::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:39,002::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:39,002::__init__::232::IOProcess::(_processLogs) (177) Got request for method 'readfile' >Thread-16::DEBUG::2014-10-15 07:27:39,003::__init__::232::IOProcess::(_processLogs) (177) Queuing response >Thread-29::DEBUG::2014-10-15 07:27:39,003::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=domain1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-0000000003e1', 'REMOTE_PATH=10.66.128.104:/home/NFS1', 'ROLE=Master', 'SDUUID=55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b90f33781d11f6e6b0b3945de3167c0719a458c1'] >Thread-16::DEBUG::2014-10-15 07:27:39,003::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:39,003::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:39,003::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:39,003::__init__::232::IOProcess::(_processLogs) (178) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:39,004::__init__::232::IOProcess::(_processLogs) (178) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:39,004::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:39,004::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:39,004::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:39,004::__init__::232::IOProcess::(_processLogs) (179) Got request for method 'glob' >Thread-16::DEBUG::2014-10-15 07:27:39,005::__init__::232::IOProcess::(_processLogs) (179) Queuing response >Thread-29::DEBUG::2014-10-15 07:27:39,005::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-29::INFO::2014-10-15 07:27:39,005::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_imageNS already registered >Thread-29::INFO::2014-10-15 07:27:39,005::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4_volumeNS already registered >Thread-16::DEBUG::2014-10-15 07:27:39,005::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:39,005::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:39,006::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:39,006::__init__::232::IOProcess::(_processLogs) (180) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:27:39,006::__init__::232::IOProcess::(_processLogs) (180) Queuing response >Thread-29::DEBUG::2014-10-15 07:27:39,006::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:27:39,013::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000415808 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:27:39,014::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:39,014::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:39,014::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:39,014::__init__::232::IOProcess::(_processLogs) (181) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:27:39,015::__init__::232::IOProcess::(_processLogs) (181) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:39,015::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:39,015::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:39,015::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:39,015::__init__::232::IOProcess::(_processLogs) (182) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:39,016::__init__::232::IOProcess::(_processLogs) (182) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:39,016::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:39,016::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:39,016::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:39,016::__init__::232::IOProcess::(_processLogs) (183) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:39,016::__init__::232::IOProcess::(_processLogs) (183) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:39,017::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:39,017::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:39,017::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:39,017::__init__::232::IOProcess::(_processLogs) (184) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:39,017::__init__::232::IOProcess::(_processLogs) (184) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:42,059::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:42,059::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-171::DEBUG::2014-10-15 07:27:42,060::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:45,071::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:45,071::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-172::DEBUG::2014-10-15 07:27:45,072::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:48,084::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:48,084::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-173::DEBUG::2014-10-15 07:27:48,085::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:27:49,018::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:49,018::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:49,019::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:49,019::__init__::232::IOProcess::(_processLogs) (185) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:27:49,019::__init__::232::IOProcess::(_processLogs) (185) Queuing response >Thread-29::DEBUG::2014-10-15 07:27:49,019::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:27:49,026::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000368448 s, 1.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:27:49,027::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:49,027::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:49,027::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:49,027::__init__::232::IOProcess::(_processLogs) (186) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:27:49,027::__init__::232::IOProcess::(_processLogs) (186) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:49,028::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:49,028::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:49,029::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:49,029::__init__::232::IOProcess::(_processLogs) (187) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:49,029::__init__::232::IOProcess::(_processLogs) (187) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:49,029::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:49,029::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:49,030::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:49,030::__init__::232::IOProcess::(_processLogs) (188) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:49,030::__init__::232::IOProcess::(_processLogs) (188) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:49,030::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:49,030::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:49,030::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:49,030::__init__::232::IOProcess::(_processLogs) (189) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:49,031::__init__::232::IOProcess::(_processLogs) (189) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:51,094::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:51,095::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-174::DEBUG::2014-10-15 07:27:51,096::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:54,105::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:54,105::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-175::DEBUG::2014-10-15 07:27:54,107::task::595::Storage.TaskManager.Task::(_updateState) Task=`d471a4b1-b4f4-455b-b92d-e3b88b3f948a`::moving from state init -> state preparing >Thread-175::INFO::2014-10-15 07:27:54,107::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-175::INFO::2014-10-15 07:27:54,107::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000368448', 'lastCheck': '5.1', 'valid': True}} >Thread-175::DEBUG::2014-10-15 07:27:54,107::task::1191::Storage.TaskManager.Task::(prepare) Task=`d471a4b1-b4f4-455b-b92d-e3b88b3f948a`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000368448', 'lastCheck': '5.1', 'valid': True}} >Thread-175::DEBUG::2014-10-15 07:27:54,107::task::595::Storage.TaskManager.Task::(_updateState) Task=`d471a4b1-b4f4-455b-b92d-e3b88b3f948a`::moving from state preparing -> state finished >Thread-175::DEBUG::2014-10-15 07:27:54,107::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-175::DEBUG::2014-10-15 07:27:54,107::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-175::DEBUG::2014-10-15 07:27:54,108::task::993::Storage.TaskManager.Task::(_decref) Task=`d471a4b1-b4f4-455b-b92d-e3b88b3f948a`::ref 0 aborting False >Thread-175::ERROR::2014-10-15 07:27:54,109::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-175::DEBUG::2014-10-15 07:27:54,109::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:54,121::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:54,122::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-176::DEBUG::2014-10-15 07:27:54,122::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:27:57,193::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:27:57,193::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-177::DEBUG::2014-10-15 07:27:57,194::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:27:59,031::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:59,032::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:59,032::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:59,032::__init__::232::IOProcess::(_processLogs) (190) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:27:59,032::__init__::232::IOProcess::(_processLogs) (190) Queuing response >Thread-29::DEBUG::2014-10-15 07:27:59,033::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:27:59,039::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000361964 s, 1.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:27:59,040::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:59,040::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:59,040::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:59,040::__init__::232::IOProcess::(_processLogs) (191) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:27:59,041::__init__::232::IOProcess::(_processLogs) (191) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:59,041::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:59,042::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:59,042::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:59,042::__init__::232::IOProcess::(_processLogs) (192) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:59,042::__init__::232::IOProcess::(_processLogs) (192) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:59,042::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:59,043::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:59,043::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:59,043::__init__::232::IOProcess::(_processLogs) (193) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:59,043::__init__::232::IOProcess::(_processLogs) (193) Queuing response >Thread-16::DEBUG::2014-10-15 07:27:59,043::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:27:59,043::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:27:59,044::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:27:59,044::__init__::232::IOProcess::(_processLogs) (194) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:27:59,044::__init__::232::IOProcess::(_processLogs) (194) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:28:00,200::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:28:00,201::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-178::DEBUG::2014-10-15 07:28:00,202::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:28:03,211::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:28:03,211::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-179::DEBUG::2014-10-15 07:28:03,212::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:28:06,222::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:28:06,223::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-180::DEBUG::2014-10-15 07:28:06,224::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:28:09,044::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:28:09,045::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:28:09,045::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:28:09,045::__init__::232::IOProcess::(_processLogs) (195) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:28:09,045::__init__::232::IOProcess::(_processLogs) (195) Queuing response >Thread-29::DEBUG::2014-10-15 07:28:09,046::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:28:09,053::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000484319 s, 948 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:28:09,053::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:28:09,054::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:28:09,054::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:28:09,054::__init__::232::IOProcess::(_processLogs) (196) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:28:09,054::__init__::232::IOProcess::(_processLogs) (196) Queuing response >Thread-16::DEBUG::2014-10-15 07:28:09,055::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:28:09,055::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:28:09,055::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:28:09,055::__init__::232::IOProcess::(_processLogs) (197) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:28:09,055::__init__::232::IOProcess::(_processLogs) (197) Queuing response >Thread-16::DEBUG::2014-10-15 07:28:09,056::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:28:09,056::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:28:09,056::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:28:09,056::__init__::232::IOProcess::(_processLogs) (198) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:28:09,056::__init__::232::IOProcess::(_processLogs) (198) Queuing response >Thread-16::DEBUG::2014-10-15 07:28:09,056::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:28:09,056::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:28:09,057::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:28:09,057::__init__::232::IOProcess::(_processLogs) (199) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:28:09,057::__init__::232::IOProcess::(_processLogs) (199) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:28:09,233::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:28:09,234::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-181::DEBUG::2014-10-15 07:28:09,235::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b8aee98-2c0b-468f-9a81-73584480f1c9`::moving from state init -> state preparing >Thread-181::INFO::2014-10-15 07:28:09,235::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-181::INFO::2014-10-15 07:28:09,235::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000484319', 'lastCheck': '0.2', 'valid': True}} >Thread-181::DEBUG::2014-10-15 07:28:09,235::task::1191::Storage.TaskManager.Task::(prepare) Task=`9b8aee98-2c0b-468f-9a81-73584480f1c9`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000484319', 'lastCheck': '0.2', 'valid': True}} >Thread-181::DEBUG::2014-10-15 07:28:09,235::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b8aee98-2c0b-468f-9a81-73584480f1c9`::moving from state preparing -> state finished >Thread-181::DEBUG::2014-10-15 07:28:09,235::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-181::DEBUG::2014-10-15 07:28:09,235::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-181::DEBUG::2014-10-15 07:28:09,236::task::993::Storage.TaskManager.Task::(_decref) Task=`9b8aee98-2c0b-468f-9a81-73584480f1c9`::ref 0 aborting False >Thread-181::ERROR::2014-10-15 07:28:09,236::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-181::DEBUG::2014-10-15 07:28:09,237::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:28:09,246::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:28:09,247::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-182::DEBUG::2014-10-15 07:28:09,247::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:28:12,323::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:28:12,324::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-183::DEBUG::2014-10-15 07:28:12,325::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:28:15,334::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:28:15,335::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-184::DEBUG::2014-10-15 07:28:15,336::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:28:18,347::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:28:18,347::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-185::DEBUG::2014-10-15 07:28:18,348::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-16::DEBUG::2014-10-15 07:28:19,058::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:28:19,058::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:28:19,058::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:28:19,058::__init__::232::IOProcess::(_processLogs) (200) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:28:19,059::__init__::232::IOProcess::(_processLogs) (200) Queuing response >Thread-29::DEBUG::2014-10-15 07:28:19,059::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.66.128.104:_home_NFS1/55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-29::DEBUG::2014-10-15 07:28:19,066::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n459 bytes (459 B) copied, 0.000410262 s, 1.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-10-15 07:28:19,066::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:28:19,067::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:28:19,067::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:28:19,067::__init__::232::IOProcess::(_processLogs) (201) Got request for method 'statvfs' >Thread-16::DEBUG::2014-10-15 07:28:19,067::__init__::232::IOProcess::(_processLogs) (201) Queuing response >Thread-16::DEBUG::2014-10-15 07:28:19,068::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:28:19,068::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:28:19,068::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:28:19,068::__init__::232::IOProcess::(_processLogs) (202) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:28:19,069::__init__::232::IOProcess::(_processLogs) (202) Queuing response >Thread-16::DEBUG::2014-10-15 07:28:19,069::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:28:19,069::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:28:19,069::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:28:19,069::__init__::232::IOProcess::(_processLogs) (203) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:28:19,069::__init__::232::IOProcess::(_processLogs) (203) Queuing response >Thread-16::DEBUG::2014-10-15 07:28:19,070::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-16::DEBUG::2014-10-15 07:28:19,070::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-10-15 07:28:19,070::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-10-15 07:28:19,070::__init__::232::IOProcess::(_processLogs) (204) Got request for method 'access' >Thread-16::DEBUG::2014-10-15 07:28:19,070::__init__::232::IOProcess::(_processLogs) (204) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:28:21,360::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:28:21,361::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-186::DEBUG::2014-10-15 07:28:21,362::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:28:24,372::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:28:24,373::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-187::DEBUG::2014-10-15 07:28:24,374::task::595::Storage.TaskManager.Task::(_updateState) Task=`0d0376f8-fa91-4658-8314-ab11a7e5d0f5`::moving from state init -> state preparing >Thread-187::INFO::2014-10-15 07:28:24,374::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-187::INFO::2014-10-15 07:28:24,374::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000410262', 'lastCheck': '5.3', 'valid': True}} >Thread-187::DEBUG::2014-10-15 07:28:24,374::task::1191::Storage.TaskManager.Task::(prepare) Task=`0d0376f8-fa91-4658-8314-ab11a7e5d0f5`::finished: {u'55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000410262', 'lastCheck': '5.3', 'valid': True}} >Thread-187::DEBUG::2014-10-15 07:28:24,374::task::595::Storage.TaskManager.Task::(_updateState) Task=`0d0376f8-fa91-4658-8314-ab11a7e5d0f5`::moving from state preparing -> state finished >Thread-187::DEBUG::2014-10-15 07:28:24,375::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-187::DEBUG::2014-10-15 07:28:24,375::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-187::DEBUG::2014-10-15 07:28:24,375::task::993::Storage.TaskManager.Task::(_decref) Task=`0d0376f8-fa91-4658-8314-ab11a7e5d0f5`::ref 0 aborting False >Thread-187::ERROR::2014-10-15 07:28:24,376::API::1687::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1674, in _getHaInfo > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 217, in get_local_host_id > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in __init__ > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 69, in _load >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-187::DEBUG::2014-10-15 07:28:24,376::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-15 07:28:24,389::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-15 07:28:24,390::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-188::DEBUG::2014-10-15 07:28:24,391::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >MainThread::DEBUG::2014-10-15 07:28:25,464::vdsm::58::vds::(sigtermHandler) Received signal 15 >MainThread::DEBUG::2014-10-15 07:28:25,464::protocoldetector::135::vds.MultiProtocolAcceptor::(stop) Stopping Acceptor >MainThread::INFO::2014-10-15 07:28:25,464::__init__::563::jsonrpc.JsonRpcServer::(stop) Stopping JsonRPC Server >Thread-16::ERROR::2014-10-15 07:28:25,465::__init__::317::IOProcessClient::(_communicate) IOProcess failure >Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 272, in _communicate >Exception: FD closed >MainThread::INFO::2014-10-15 07:28:25,465::vmchannels::188::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-10-15 07:28:25,466::momIF::91::MOM::(stop) Shutting down MOM >MainThread::DEBUG::2014-10-15 07:28:25,466::task::595::Storage.TaskManager.Task::(_updateState) Task=`ecf2e3e7-2e71-4873-96ca-3ca5f222c047`::moving from state init -> state preparing >MainThread::INFO::2014-10-15 07:28:25,466::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Detector thread::DEBUG::2014-10-15 07:28:25,467::protocoldetector::106::vds.MultiProtocolAcceptor::(_cleanup) Cleaning Acceptor >Thread-16::DEBUG::2014-10-15 07:28:25,467::__init__::159::IOProcessClient::(_run) Starting IOProcess... >Thread-11::DEBUG::2014-10-15 07:28:25,467::storageServer::706::Storage.ConnectionMonitor::(_monitorConnections) Monitoring stopped >VM Channels Listener::INFO::2014-10-15 07:28:26,292::vmchannels::183::vds::(run) VM channels listener thread has ended. >MainThread::INFO::2014-10-15 07:28:26,645::domainMonitor::137::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::INFO::2014-10-15 07:28:26,645::domainMonitor::114::Storage.DomainMonitor::(stopMonitoring) Stop monitoring 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-29::DEBUG::2014-10-15 07:28:26,645::domainMonitor::192::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 >Thread-29::INFO::2014-10-15 07:28:26,645::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 (id: 2) >Thread-38::INFO::2014-10-15 07:28:26,774::storage_mailbox::509::Storage.MailBox.HsmMailMonitor::(run) HSM_MailboxMonitor - Incoming mail monitoring thread stopped, clearing outgoing mail >Thread-38::INFO::2014-10-15 07:28:26,774::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/usr/bin/dd', 'of=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=16'] >Thread-38::DEBUG::2014-10-15 07:28:26,775::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /usr/bin/dd of=/rhev/data-center/00000002-0002-0002-0002-0000000003e1/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=16 (cwd None) >Thread-38::DEBUG::2014-10-15 07:28:26,905::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.125547 s, 32.6 kB/s\n'; <rc> = 0 >Thread-29::DEBUG::2014-10-15 07:28:28,646::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain 55e12fa3-f7ee-4f13-96f8-ed5dd690f7d4 released successfully (id: 2) >MainThread::DEBUG::2014-10-15 07:28:28,646::taskManager::90::Storage.TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2014-10-15 07:28:28,647::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2014-10-15 07:28:28,647::task::1191::Storage.TaskManager.Task::(prepare) Task=`ecf2e3e7-2e71-4873-96ca-3ca5f222c047`::finished: None >MainThread::DEBUG::2014-10-15 07:28:28,647::task::595::Storage.TaskManager.Task::(_updateState) Task=`ecf2e3e7-2e71-4873-96ca-3ca5f222c047`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-15 07:28:28,647::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-15 07:28:28,647::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-15 07:28:28,647::task::993::Storage.TaskManager.Task::(_decref) Task=`ecf2e3e7-2e71-4873-96ca-3ca5f222c047`::ref 0 aborting False >MainThread::INFO::2014-10-15 07:28:28,647::vdsm::137::vds::(run) VDSM main thread ended. Waiting for 12 other threads... >MainThread::INFO::2014-10-15 07:28:28,647::vdsm::141::vds::(run) <Thread(libvirtEventLoop, started daemon 139904430155520)> >MainThread::INFO::2014-10-15 07:28:28,647::vdsm::141::vds::(run) <WorkerThread(Thread-7, started daemon 139904164017920)> >MainThread::INFO::2014-10-15 07:28:28,647::vdsm::141::vds::(run) <WorkerThread(Thread-10, started daemon 139903660717824)> >MainThread::INFO::2014-10-15 07:28:28,648::vdsm::141::vds::(run) <WorkerThread(Thread-1, started daemon 139904421762816)> >MainThread::INFO::2014-10-15 07:28:28,648::vdsm::141::vds::(run) <Thread(Thread-190, started daemon 139903389492992)> >MainThread::INFO::2014-10-15 07:28:28,648::vdsm::141::vds::(run) <WorkerThread(Thread-9, started daemon 139904147232512)> >MainThread::INFO::2014-10-15 07:28:28,648::vdsm::141::vds::(run) <WorkerThread(Thread-8, started daemon 139904155625216)> >MainThread::INFO::2014-10-15 07:28:28,648::vdsm::141::vds::(run) <WorkerThread(Thread-3, started daemon 139904197588736)> >MainThread::INFO::2014-10-15 07:28:28,648::vdsm::141::vds::(run) <WorkerThread(Thread-6, started daemon 139904172410624)> >MainThread::INFO::2014-10-15 07:28:28,648::vdsm::141::vds::(run) <WorkerThread(Thread-5, started daemon 139904180803328)> >MainThread::INFO::2014-10-15 07:28:28,648::vdsm::141::vds::(run) <WorkerThread(Thread-4, started daemon 139904189196032)> >MainThread::INFO::2014-10-15 07:28:28,648::vdsm::141::vds::(run) <_MainThread(MainThread, started 139904666838848)> >MainThread::INFO::2014-10-15 07:28:28,648::vdsm::141::vds::(run) <WorkerThread(Thread-2, started daemon 139904413370112)>
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1153444
: 947403 |
949733
|
949896